How Does TLS 1.3 Work? Protocols, Key Attacks, Security Practices

If your project requires software to connect with other products over the internet, then securing transferred data is a must. Left unprotected, data transmissions become lucrative targets for malicious actors. And as a result, your entire infrastructure could be at risk.

The best way to mitigate threats like data theft and hacks is by preventing them. One way to do so is to involve a cybersecurity engineer who can implement and verify protection of network communications.

In this article, you’ll discover crucial information about the most modern encryption standard at the time of writing — Transport Layer Security (TLS) version 1.3. You’ll discover how TLS 1.3 works and find a list of key types of attacks to expect and ways to prevent them.

This article will be useful for project and development leaders wanting to enhance their software protection and looking for expertise in network security.

Why use TLS 1.3?

Secure data transmission is a critical aspect of interactions between HTTPS clients and servers. Before clients and servers exchange any data, it first has to be encrypted according to a reliable cryptographic standard. Such standards help to establish secure connections between applications by encrypting any communication packets sent between a client and a server.

Transport Layer Security (TLS) is a family of protocols designed by the Internet Engineering Task Force to secure client–server communications over a network. TLS combines symmetric and asymmetric encryption mechanisms to help a client and server establish a reliable communication channel and safely exchange data. 

Currently, there are four protocols in this family, but only TLS 1.3 is recommended for implementation. It focuses on enhancing security, reducing complexity, and improving the performance of software communications.

Compared to other encryption standards, here’s why use TLS 1.3:

TLS consists of several key protocols and lots of security configurations. To truly protect your data and experience these benefits, your team needs to have a deep understanding of cryptography as well as experience implementing and configuring TLS 1.3.

Let’s examine how to use TLS 1.3 and its protocols. We’ll analyze how a client and server make their connection secure before sharing any data.

The handshake protocol

TLS uses a handshake protocol to establish a secure connection. The handshake begins with the client initiating the process by sending a ClientHello message. Upon receiving ClientHello, the server responds with ServerHello.

Following initial contact, the client and server exchange certificates and parameters to agree on encryption for future messages. This communication ends with ServerFinished and ClientFinished messages. From here on, all data transmitted between the client and the server is encrypted using session keys.

An optional feature in TLS 1.3 is zero round trip time (0-RTT) data. If the client has a preshared key from a previous session, it can send 0-RTT data along with ClientHello to send application data immediately, reducing latency. However, the server processes 0-RTT data only if it accepts the pre-shared key (PSK), and this feature comes with some security tradeoffs, such as the potential for replay attacks.

Here’s how this exchange looks:

Here’s an example of a ClientHello message:

ClientHello {
  Protocol Version: TLS 1.3
  Random: <32 bytes of random data>
  Session ID: <session ID or empty>
  Cipher Suites: [TLS_AES_128_GCM_SHA256, TLS_CHACHA20_POLY1305_SHA256, ...]
  Compression Methods: [null]
  Extensions: {
    Supported Versions: [TLS 1.3, TLS 1.2]
    Supported Groups: [X25519, secp256r1, secp384r1]
    Key Share: [
      Group: X25519, Key Exchange Data: <client's public key share>
    ]
    Signature Algorithms: [ecdsa_secp256r1_sha256, rsa_pss_rsae_sha256, ...]
    Pre-Shared Key (PSK) Modes: [psk_dhe_ke]
    Server Name Indication (SNI): example.com
    Application-Layer Protocol Negotiation (ALPN): [http/1.1, h2]
    Signature Algorithms Cert: [ecdsa_secp256r1_sha256, rsa_pss_rsae_sha256, ...]
    Supported Points Formats: [uncompressed]
    Cookie: <optional cookie data>
  }
}

In this message, the client communicates: 

• Protocol version — the highest version of TLS that the client supports. This field indicates TLS 1.3, ensuring that the server is aware of the client’s ability to use the latest protocol version.
• Client random — a 32-byte random value generated by the client. It’s required to create session keys and ensure the freshness of the handshake, preventing replay attacks.
• Session ID — a field included for compatibility with previous TLS versions.
• Cipher suites — a list of supported cipher suites, each representing a combination of encryption algorithms and key exchange mechanisms. TLS 1.3 mostly uses authenticated encryption with associated data (AEAD) cipher suites, such as AES-GCM and ChaCha20-Poly1305.
• Compression methods — another field for backward compatibility. It’s set to null to indicate that no compression is supported.
• Extensions — the client’s additional capabilities and preferences. 

In our example, the client’s message contains all major extensions in TLS 1.3. Let’s see what they mean:

• Supported versions — the list of TLS versions that the client supports, allowing the client and server to use the highest mutually supported version.
• Supported groups — the elliptic curve groups supported by the client for key exchange. This helps in determining the group for Diffie-Hellman (DH) or Elliptic Curve Diffie-Hellman (ECDH) key exchanges.
• Key share — the client’s public key shares for supported key exchange methods that a server can use to exchange keys without requiring an additional round trip.
• Signature algorithms — the list of digital signature algorithms that the client supports for verifying certificates and signing messages.
• Pre-shared key (PSK) modes — the modes supported by the client if it wants to resume a session using a previously shared key.
• Server name indication (SNI) — the hostname that the client is trying to connect to, which is particularly useful for servers hosting multiple domains.
• Application-layer protocol negotiation (ALPN) — a list of application protocols supported by the client, allowing the server to select the appropriate one.
• Signature algorithms cert — the list of signature algorithms that the client supports for certificate signatures.
• Supported points formats — the formats for elliptic curve point encoding supported by the client.
• Cookie — additional data for stateless client authentication scenarios.

Now, we can take a look at the ServerHello response to this message:

ServerHello {
  Protocol Version: TLS 1.3
  Random: <32 bytes of random data>
  Session ID: <empty or session ID for compatibility>
  Cipher Suite: TLS_AES_128_GCM_SHA256
  Compression Method: null
  Extensions: {
    Key Share: {
      Group: X25519, Key Exchange Data: <server's public key share>
    }
    Pre-Shared Key (PSK) Identity: <selected PSK identity (optional)>
    Supported Versions: [TLS 1.3]
    Server Name Indication (SNI): example.com (optional)
    Application-Layer Protocol Negotiation (ALPN): h2 (optional)
  }
}

Let’s analyze this message:

• Protocol version — the server confirms the protocol version to be used for the session. In TLS 1.3, this field specifies TLS 1.3, ensuring compatibility with the client’s capabilities.
• Server random — a 32-byte random value generated by the server. Like the client random, it’s used to create session keys and ensure the uniqueness of each session.
• Session ID — this field is included for compatibility with earlier versions of TLS.
• Cipher suite — the server selects one cipher suite from the list provided in ClientHello. This chosen cipher suite defines the encryption algorithm and mode of operation to use for the session.
• Compression method — a field included for backward compatibility.
• Extensions — a list of extensions that provide additional parameters and support for specific functionalities. 

Key extensions for a server in TLS 1.3 include:

• Key share — the server’s public key share for the selected key exchange group. This extension allows the client to complete the key exchange process.
• PSK identity — the PSK identity selected by the server for resuming a previous session without a full handshake.
• Supported versions — the highest supported TLS version mutually agreed upon.
• SNI — if the server supports multiple domains, it may use this extension to confirm the specific domain being accessed.
• ALPN — one of the application protocols proposed by the client.

Handshake variations

TLS 1.3 provides several mechanisms to establish a connection in case the initial handshake was flawed. For example, if a client hasn’t provided an appropriate key_share extension, the server must correct the mismatch with the HelloRetryRequest message to reinitiate the handshake.

After the handshake, the server derives a unique PSK identity and can send it to the client. The client can reuse this identity to establish a cryptographically secure connection without repeating the handshake. This mechanism replaces now obsolete session IDs and session tickets from previous TLS versions. PSKs can be used with (EC)DHE key exchange or without it, which is less secure.

The server doesn’t need to send its certificate or CertificateVerify message, since authentication is done via PSK. The client can add a key_share extension in case the server declines PSK and chooses to do the full handshake. The server can also add a key_share extension with (EC)DHE parameters for forward secrecy.

When a client and server share PSK, they can send early data in 0-RTT mode. Data sent this way has weaker security properties than data sent over a normal connection because 0-RTT data isn’t encrypted without forward secrecy, and there’s no guarantee of non-replay between connections. It’s impossible to duplicate 0-RTT data within a connection, and a potential attacker can’t make 0-RTT data appear to be normal 1-RTT data because it is protected with different keys.

When the handshake is successfully completed, the client and server can exchange information via a protected channel. But before sending application data, they need to choose data encryption algorithms they both support. This is done by exchanging a set of parameters — more on this in the following section.

Related project

Developing a Custom Secrets Management Desktop Application for Secure Password Sharing and Storage

Explore the story of our client who needed help developing an internal app for secrets management. We collaborated on a solution that helped them improve the company’s overall security score by 30%.

Project details

Negotiating encryption parameters

A server starts this exchange by sending several of its own parameters in a series of messages:

• Encrypted Extensions contains additional parameters for the handshake. These parameters are encrypted using the shared key derived from the initial key exchange. This message ensures that only the client and server can read the parameters, enhancing the security of communication.
• Server Certificate contains the server’s certificate chain, which provides the server’s public key and proves its identity. The certificate typically includes the server’s public key, information about the server, and a digital signature from a trusted Certificate Authority (CA).
• Certificate Verify contains a digital signature for the handshake messages exchanged so far. The server signs this data with its private key, and the client can verify the signature using the server’s public key from the certificate.
• Server Finished includes a hash of the entire handshake up to this point. This message is encrypted with the session key derived from the key exchange. It confirms that the server has completed its part of the handshake and that it wasn’t tampered with.

The client processes these messages and responds with:

• Client key share, if it wasn’t included in the initial ClientHello. This key share is used in the key exchange process to derive the shared secret key.
• PSK in case of a resumed session. The client sends the selected PSK identity and uses the PSK to derive session keys.
• Client Certificate, which is an optional message if client authentication is required. This certificate chain is identical to the server’s chain used before.
• Certificate Verify, which is another optional message that can prove ownership of the private key corresponding to the public key in the client’s certificate. The client signs this data with its private key, and the server can verify the signature using the client’s public key from the certificate.
• Client Finished includes a hash of the entire handshake up to this point, encrypted with the session key derived from the key exchange. This message confirms that the client has completed its part of the handshake and that it has not been tampered with.

When the parameter exchange is finished, both parties have all the necessary information to derive the shared secret keys. The session keys are derived using the DHE or ECDH key exchange, which ensures perfect forward secrecy.

Both the client and server combine their respective key shares to compute the shared secret. This shared secret is then used to derive the session keys using a key derivation function specified in the cipher suite. Then, the client and server calculate encryption keys and initialization vectors (IVs). 

Once the session keys are established, the client and server can securely exchange data. Every piece of data is encrypted with the session keys, ensuring confidentiality, integrity, and authenticity. If either party needs to update keys during the session, they can perform a key update without restarting the handshake process.

Read also

9 Web Application Security Best Practices: Real-Life Examples from Apriorit Experts

Protect your app from the majority of web risks and attacks. Read the full text to find out what web security practices to implement and how.

Learn more

Record layer

The TLS record layer ensures secure communication by managing how data is transmitted between a client and server. It fragments data into records, encrypts them to ensure confidentiality, and applies integrity checks to prevent tampering. Until the client and server negotiate encryption parameters, they send plaintext messages. However, once encryption parameters are established, messages undergo protection conversion.

Each fragment contains no more than 16,384 (2¹⁴) bytes. Each message contains the content type, legacy version set to TLSv1.2, fragment length, and fragment itself. Records are handled differently depending on their content type.

Handshake messages can be sent subsequently in different records. Handshake messages don’t mix with key changes: they align to record boundaries. There can be no fragments of zero-length messages.

Alert messages aren’t fragmented across different records, and each record can contain only one alert.

Application data messages are always protected, and the TLS protocol can’t determine their contents. In this case, it’s allowed to send zero-length messages, and fragments can be split across multiple records or united in a single one.

Record protection

TLS 1.3 adds several security measures to the records it processes. All ciphers comply with the authenticated encryption with associated data (AEAD) scheme, which establishes unified encryption and authentication. Each protected message has a plaintext header and encrypted body with type and optional padding of zeros. Messages shouldn’t be longer than 255 bytes. The record version field is set to TLSv1.2  for middlebox compatibility.

AEAD algorithms use several elements to form an authentication check:

• Write key (client_write_key or server_write_key)
• Nonce, formed from the sequence number
• Write IVs
• Additional data from the record input

TLS establishes separate record sequence numbers for reading and writing data, adding an appropriate number once a record is read or written. This is a 64-bit number, and if the limit is exceeded, the connection should either be rekeyed or terminated. 

TLS also limits the amount of data that can be safely encrypted by a given set of keys. The limit varies between different ciphers.

The minimum and maximum length of a nonce depends on the selected AEAD algorithm, but its upper limit shouldn’t be lower than 8 bytes. The nonce itself is formed from a sequence number in network order padded to the left of IV length with zeros. This value is then XORed with IV.

Padding helps hide traffic size from a potential observer by masking it. The padding doesn’t affect size limitations, so it’s added to all records.

Alert protocol

Alerts in TLS are encrypted messages that indicate issues and errors occurring during communication. There are two types of alerts: 

• Closure alerts are sent by a client or server to indicate connection termination. When an entity receives this alert, it should inform the application about the end of connection. There are two types of closure alerts: default close_notify for termination after a handshake and user_canceled for termination during a handshake.
• Error alerts indicate unplanned connection termination because of some issue, such as a missing extension or unknown PSK identity. All unknown alerts are treated as errors. Upon sending and receiving an error, peers clear all cached secrets and keys, except PSKs associated with session tickets.

Though TLS 1.3 provides many protocols and mechanisms for securing data, it’s still susceptible to attacks and threats. In the next section, we take a look at key TLS 1.3 challenges from a security standpoint, attack vectors, and practices to enhance TLS protection.

Related project

Custom Cybersecurity Solution Development: From MVP to Support and Maintenance

Check out our case study about collaborating on a platform that detects suspicious events by monitoring user actions. We helped them test their business idea, turn the MVP into a release-ready product, and ensure compliance with regional and industry-specific requirements.

Project details

8 key attacks against TLS 1.3 and how to stop them

TLS 1.3 is one of the most common security protocols for web communications. As of May 2024, 70% of all websites monitored by SSL Labs supported TLS 1.3. Despite the many security benefits of TLS 1.3, it has some vulnerabilities that are well-known to malicious actors. 

Here are the key types of attacks hackers can use to steal data or listen to TLS-protected communications:

Let’s examine each attack vector, providing examples and ways to protect your web application from them.

1. Side-channel attacks

Side-channel attacks on TLS 1.3 exploit indirect information leaks, such as information on timing, power consumption, or electromagnetic signals, to infer sensitive data. These attacks don’t target the encryption itself but rather weaknesses in its implementation.

This exploit vector includes:

• Timing attacks. Attackers can infer sensitive information by measuring the time taken to encrypt and decrypt data. For example, slight variations in processing time can reveal information about private keys used for encryption. This method also helps attackers to detect plaintext data. To mitigate timing attacks, use constant-time algorithms based on elliptic curve cryptography, Montgomery multiplication, and bitsliced AES that ensure cryptographic operations take the same amount of time regardless of the input data.
• Cache attacks. Attackers can exploit shared CPU caches to extract information about cryptographic operations. By analyzing cache hits and misses, they can infer key material. You can prevent such attacks by implementing cache-resistant algorithms, flushing caches before and after cryptographic operations, and isolating sensitive processes.

2. Replay attacks

Replay attacks involve an attacker intercepting valid data transmissions and maliciously repeating or delaying them to deceive a system. In these attacks, the adversary doesn’t need to decipher the data. They replicate previously captured messages to perform certain actions or gain unauthorized access.

The most common example of a replay attack is 0-RTT data replay, where an attacker captures and transmits 0-RTT data sent by the client. Replaying these records can cause repeated execution of certain operations. 

To prevent such attacks, make sure to implement replay protection mechanisms on your server to detect and reject replayed 0-RTT data. For example, you can create a list of used nonces and timestamps or limit 0-RTT data to idempotent operations like GET requests.

3. Man-in-the-middle (MitM) attacks

An MitM attack occurs when an attacker secretly intercepts and potentially alters the communication between two parties who believe they are directly communicating with each other. The attacker positions themselves between the client and server, gaining access to sensitive information like login credentials, financial data, or private messages.

TLS-protected communication can be susceptible to certificate forgery. This attack happens when attackers create a fraudulent certificate or compromise a certificate authority, allowing them to intercept and alter communications. You can stop forgery by using Certificate Transparency to detect rogue certificates, enforcing strict certificate validation procedures, and employing techniques such as HTTP Public Key Pinning.

4. Key exchange attacks

Key exchange attacks target the process of exchanging cryptographic keys between parties, attempting to intercept or manipulate the keys to compromise encrypted communications. These attacks often exploit weaknesses in protocols or implementations, enabling an attacker to gain access to the shared secret used for encryption.

TLS 1.3 mandates the use of strong key exchange mechanisms like Elliptic Curve Diffie-Hellman to prevent such attacks. You can enhance the protection of key exchanges by only using well-vetted, secure libraries for all cryptographic operations. 

At the time of writing, the only way to compromise key exchange in TLS 1.3 is by using quantum computing. While quantum computing capabilities aren’t advanced and accessible enough for hackers at the moment, this will change in the future. That’s why organizations should be ready to implement post-quantum cryptography to protect their software from new ways to intercept keys that will come with the advancement of quantum computing.

Read also

Building a Transparent Encryption Framework to Secure Android Applications: A Step-by-Step Guide

Gain insights from Apriorit’s experience in building a reliable encryption framework. We walk you through planning the framework, implementing key features, creating a cryptographic library, and writing wrappers.

Learn more

5. Denial of service (DoS) attacks

DoS attacks aim to disrupt the availability of a server, network, or service by overwhelming it with requests, consuming resources, or exploiting vulnerabilities to crash it. These attacks prevent legitimate users from accessing the targeted system. A distributed denial of service (DDoS) attack amplifies the impact of a DoS attack by using multiple systems, which are often part of a botnet.

Using DoS methods, attackers can try to perform:

• Resource exhaustion. Attackers initiate numerous TLS handshakes to exhaust server resources, leading to denial of service. You can prevent this by implementing rate limiting, using CAPTCHAs to distinguish legitimate users, and employing DDoS protection services.
• Protocol exploits. Malicious actors can find exploits in the TLS protocol or its implementation to crash servers or degrade their performance. The only way to prevent such exploits is to continuously monitor for vulnerabilities, apply new security patches, and use intrusion detection systems to identify and mitigate exploit attempts.

6. Session hijacking

Session hijacking occurs when an attacker takes over a legitimate user’s session with a service, gaining unauthorized access to the user’s data or actions. This is typically done through:

• Token stealing. Attackers steal session tokens or cookies that allow them to hijack an active session. You can prevent this by ensuring the secure transmission and storage of session tokens, using secure and HTTP-only flags for cookies, and implementing short-lived tokens with frequent re-authentication.
• Session fixation. Attackers set a session ID that is known to them before the user logs in, allowing them to hijack the session afterward. To mitigate this type of attack, regenerate session IDs upon user authentication and enforce strict session management practices.

7. Forward secrecy compromise

Forward secrecy ensures that even if an attacker obtains the server’s private key, they cannot decrypt past communications because each session uses a unique ephemeral key. An attacker can try gaining access to long-term keys to decrypt past sessions that did not use forward secrecy.

TLS 1.3 enforces forward secrecy using ephemeral key exchanges. To protect long-term keys, make sure to adopt proper key management and secure storage practices.

8. Implementation flaws

TLS 1.3 enforces many security checks and mechanisms to improve the protection of data sent over the internet. However, human errors and mistakes can significantly reduce TLS’s effectiveness and create vulnerabilities that hackers can exploit. 

For example, developers in charge of TLS implementation can use outdated or vulnerable TLS libraries like OpenSSL or BoringSSL. They can also misconfigure TLS security by, for example, enabling insecure fallback options or improperly configuring key exchange parameters.

Though it isn’t possible to completely eliminate the risk of human errors, you can reduce their probability with the following practices:

• Regularly update cryptographic libraries to the latest versions
• Perform thorough code reviews and audits
• Follow TLS configuration best practices, including disabling insecure protocols and cipher suites
• Enable only necessary TLS features
• Regularly review configuration settings

Conclusion

Implementing TLS in your application is an absolute must. With it, you can secure application and user data, significantly enhancing your software’s cybersecurity posture and general reliability. However, thoughtful TLS implementation requires a deep understanding of TLS’s internal protocols and mechanisms.

When working on the cybersecurity of any web or cloud solution, the Apriorit team carefully reviews how it can benefit from TLS, which security options we need to enable, and how to further protect sensitive data. Our end goal is to ensure that our clients can trust their solutions with any data.