123eworld Knowledge Hub → Transactional SMS API → Page 199
Transactional SMS API Unicode and Encoding: GSM-7, Unicode SMS, Length and Concatenation
Developer reference guide for SMS Unicode and encoding.
SMS is encoding-sensitive
SMS length is not determined only by the number of visible characters. The character set can affect encoding and therefore the number of segments. Developers should calculate encoding after the final message has been rendered, including runtime variables. This is especially important for OTPs, invoices and alerts where an unexpected character can change cost and segmentation.
GSM-7
GSM-7 supports a broad set of commonly used characters efficiently, but it is not identical to ordinary ASCII. Some characters use special extension rules. A correct segment calculator must follow the actual SMS encoding rules rather than simply counting Unicode code points.
Unicode messages
When a message contains characters outside the supported GSM-7 repertoire, Unicode encoding may be required. A single unexpected character can change the segment calculation for the whole message. Customer-facing tools should make this visible before sending so developers can understand why a message uses more segments than expected.
Concatenated SMS
Long messages are divided into multiple segments and reassembled by the receiving handset. Concatenation metadata consumes part of the available payload. The maximum text capacity per segment therefore differs between single-segment and concatenated messages and between encoding modes.
API design
A useful SMS API can return normalized length, detected encoding and estimated segment count before submission. This can be exposed as a validation or preview operation. The final billing calculation should still be based on the actual submission rules used by the provider route.
Unicode normalization
Do not silently rewrite customer content merely to save segments. Normalization can change meaning, spelling or identifiers. If the platform chooses to normalize a specific character class, document the behaviour and provide a way to detect the normalized result.
Template variables
Encoding decisions must happen after variable substitution. A template containing only Latin characters may switch to Unicode when a name or transaction description contains an accented character, regional script or emoji. Template previews should therefore support representative variable values.
Emoji
Emoji can require Unicode encoding and may have device-dependent rendering behaviour. Developers should not assume that an emoji consumes one simple character position or will look identical on every handset. For transactional content, plain text may be preferable when predictability matters.
Billing
Segment count affects usage. Billing records should preserve the segment count and encoding decision used for each provider attempt. This allows support to explain a higher-than-expected charge and gives customers confidence that the calculation is deterministic.
Testing
Test GSM-7 text, extension characters, accented text, Indian scripts, mixed-language messages, emoji, line breaks and long content. Include boundary cases around segment limits because off-by-one errors are common in encoding implementations.
Monitoring
Monitor sudden changes in average segments per message. A template or application change can accidentally introduce Unicode into thousands of messages, increasing cost and provider traffic. A segment-count anomaly is therefore both a cost and an engineering signal.
Implementation takeaway
Encoding should be treated as part of message validation, pricing, provider submission and observability. A robust SMS platform makes the encoding decision explicit instead of leaving developers to discover segmentation through failed or expensive sends.
Segment calculation algorithm
A production segment calculator should operate on the exact rendered text and a well-defined encoding table. It should account for GSM-7 extension characters, Unicode encoding, concatenation headers and provider-specific limits where applicable. Keep the calculation logic centralized so the preview API, billing system and provider adapter do not each implement slightly different rules.
Boundary testing
Most encoding defects occur at boundaries. Test text exactly at the single-segment limit, one character above it, and several characters around concatenated limits. Test the same boundaries after adding a runtime variable. A change from GSM-7 to Unicode can move a message across multiple thresholds, so boundary tests should cover both encoding modes.
Indian-language messaging
Messages in Marathi, Hindi, Bengali, Tamil and other scripts normally require Unicode handling. A platform serving Indian businesses should make this a first-class scenario rather than treating Unicode as an unusual exception. The UI and API should expose the detected encoding and estimated segments so developers can design content with predictable cost.
Encoding and provider adapters
The platform should calculate logical message characteristics before routing, while provider adapters translate them into provider-specific parameters. If one provider reports segments differently, retain both the platform calculation and provider evidence. This makes billing and troubleshooting possible without allowing provider-specific behaviour to leak into every application integration.
Encoding checklist
Test multilingual templates, emojis, punctuation, line breaks, extension characters, long messages and variable substitution. Monitor segment distribution after releases. Encoding is a small technical detail with direct effects on customer experience, throughput and cost, so it deserves the same engineering discipline as authentication and retries.
Production scenario
A common production incident begins with a content change that adds one Unicode character to a previously single-segment template. Segment count increases across thousands of messages and provider usage rises unexpectedly. A preview or CI check that reports encoding and segments can catch this before release. Encoding checks should therefore be part of application testing, not only provider integration testing.
Design implication
The segment calculator should have a single authoritative implementation. If the API preview says two segments while billing records three, customers will lose confidence and support teams will struggle to reconcile usage. Keep encoding tables and calculation rules versioned so a future change can be associated with the messages affected by it.
Operational consideration
Concatenation also has a user-experience dimension. Long messages may arrive as multiple parts, and delays between parts can make a notification confusing. For critical transactional alerts, a concise single-segment message can sometimes be more valuable than a long message that contains every detail. Applications should be encouraged to link to a secure portal when more detail is required.
Developer integration note
Mixed-language content deserves explicit tests. Indian applications may combine English account labels with Hindi, Marathi or other regional text. A single non-GSM character can change the encoding mode. Test realistic combinations rather than testing each language in isolation.
Review and testing
Cost dashboards should show segments per logical message, not only total messages. A sudden rise from one segment to three segments is immediately visible in this metric. Correlate the change with template version and application release so engineers can identify the cause.
Final engineering rule
Final rule: calculate encoding from the final rendered message, expose the result, and keep one authoritative segment algorithm for preview, billing and submission.
Advanced implementation note
A preview endpoint can return encoding, segment count and normalized length without submitting the message. This is useful for application development and for content-management systems that need to show cost before activation. The preview result should clearly state that final provider behaviour can vary where route-specific rules apply.
Integration consideration
Do not treat emojis as a simple countable unit. Unicode code points, grapheme clusters and SMS encoding rules are different concepts. The SMS segment calculator should follow the messaging encoding standard rather than a user-interface character counter.
Operational safeguard
Encoding decisions should be recorded with the logical message and provider attempt. If a customer disputes usage, support can compare the rendered content, encoding mode and segment count used at submission. This evidence is much stronger than trying to recreate the calculation months later.
Architecture review
Applications that send multilingual transactional messages should include representative language samples in CI. A release that changes a template or variable source can unintentionally change encoding. Automated segment regression tests are inexpensive compared with investigating a large usage increase after deployment.
Advanced design note
Segment calculation should be deterministic across API nodes and releases. If different application versions use different encoding libraries, the same message could receive different segment estimates. Keep the encoding implementation centralized or versioned and include its revision in usage evidence when the calculation rules change.
Final production consideration
When provider-specific segmentation differs, do not hide the difference. Return the platform's logical estimate and retain provider evidence. This gives customers a stable conceptual model while still allowing support to explain route-specific billing or submission behaviour.
Continue through the 123eworld Knowledge Hub
Explore the 123eworld Knowledge Hub for practical SMS API, transactional messaging and developer architecture guides.
Visit 123eworld.com for messaging and digital communication services.