What Is Layer 1 Optical Encryption, and How Does It Differ from MACsec and IPsec?
Layer 1 optical encryption encrypts the payload of an optical transport frame in hardware, at line rate, between the two ends of a fiber link. MACsec encrypts Ethernet frames at Layer 2 one hop at a time, and IPsec encrypts IP packets at Layer 3 from one IP endpoint to another. All three put ciphertext on a standard optical signal, so a passive tap on the fiber still records a complete copy of the encrypted stream. These layers apply to WAN links, where traffic crosses fiber that other parties own and maintain. Carmel, the photonic layer security platform, works beneath all three to effectively block a harvest off the fiber.
Layer numbers come from the OSI model, which divides network communication into seven layers. Layer 1 is the physical layer, the signal on the fiber itself. Layer 2 is the data link layer, which moves Ethernet frames between neighboring devices. Layer 3 is the network layer, which routes IP packets between networks. Layers 4 to 7 are the transport, session, presentation, and application layers, and protocols such as TLS protect traffic at those higher layers.
Layer 1 optical encryption differs from MACsec and IPsec in where it sits and how much of the transmission it covers. It works on Layer 1 and encrypts the payload of an optical transport frame on one link. MACsec works on Layer 2 and encrypts Ethernet frames between neighboring devices. IPsec works on Layer 3 and encrypts IP packets between two endpoints that may be many hops apart. Each layer encrypts a different unit of traffic, so the three are not interchangeable, and many WAN designs combine combinations of them. This article defines the security of each layer and their deployment implications.
Layer 1 optical encryption on a single link
Layer 1 optical encryption works inside the optical transport network for carrying client traffic over fiber optic cables. Electronic Design describes OTN encryption as protocol-agnostic because it encrypts the payload frame regardless of the client signal. A single Layer 1 encryptor can protect the aggregated traffic of many interfaces. The cost is hardware at both ends. Both ends of the link must support the same capability.
Layer 2 - MACsec: Ethernet frames, one hop at a time
MACsec encrypts the contents of Ethernet frames at Layer 2 with AES-GCM. It works hop by hop, so each link between two neighboring devices is encrypted and then decrypted at the next device. MACsec protects each link on its own and does not protect traffic across the wider network. Traffic crossing a link that is left unsecured is exposed on that link. The Ethernet header stays in the clear so that switches can still forward frames. That design fits metro and campus Ethernet well. Across a long WAN path, every additional device is another interface to configure.
Layer 3 - IPsec: IP packets between endpoints
IPsec forms the encryption basis of many VPN products. It runs end to end between two IP endpoints, and intermediate routers cannot decrypt the packets. That independence from the underlying infrastructure lets it span WAN paths across carriers and vendors. Post-quantum migration for IPsec concerns its key exchange, since the Diffie-Hellman or elliptic-curve step that sets up each session is being supplemented, and eventually replaced, by ML-KEM.
How Do You Implement Post-Quantum Encryption for Data-in-Transit? Learn how ML-KEM replaces the key exchange step in TLS, VPNs, and IPsec.
What all three leave on the fiber
Layer 1 encryption, MACsec, and IPsec all place ciphertext on a standard optical signal, so a passive tap records a complete copy of it. The AES-256 cipher these layers commonly use is regarded as quantum-resistant, and new regulations, such as EO 14412 out of the USA, require that its key establishment also be transitioned to post-quantum methods. However, the cipher payload and the key exchange messages can still be harvested off the fiber network cable. NIST's glossary, citing CNSSI 4009, defines a passive attack as one that does not alter systems or data. Indeed, a tap of the fiber cable leaves no entry in any log and the owner may never learn a copy exists. The extracted and stored copy stays vulnerable to compromise by an adversary, for as long as the algorithms protecting it hold.
Wondering how taps are installed and how operators look for them? We break it down in - Can Fiber Optic Cables Be Tapped, and How Do Enterprises Detect It?
Carmel beside the three layers
Carmel provides harvest-proof protection at Layer 1, working on existing fiber and DWDM equipment. It blocks the harvesting step that the three encryption layers leave open.
Photonic layer security (PLS) protects the transmission by changing the light. Carmel spreads the signal across a wide optical band, shifts its phase with a key that changes every fraction of a second, and buries it beneath optical noise, so a tap captures noise and not a copy of the ciphertext. Carmel covers data in transit on WAN links, it needs a matched pair of units on each protected link, and it cannot protect traffic recorded before it was deployed.
Matching the layer to the link
The right security configuration depends on the link and on what a recording of its traffic would be worth years later. IPsec fits tunnels between sites that no single network connects, MACsec fits Ethernet links between adjacent devices, and Layer 1 optical encryption fits high-capacity optical links between data centers and carrier sites. PLS, which is also well-suited to high-capacity optical links, has the added value of blocking the harvest, not just encrypting the signal. It should be considered on links that are used to transmit sensitive long-term data, with encryption that may not outlast quantum advances. Carmel shares the C-band with other wavelengths, so encrypted alien wavelengths from other vendors can stay in service on the same fiber. Yet, while post-quantum cryptography keeps protecting the keys from eventual decryption, CyberRidge prevents that data from being harvested in the first place. Whatever can’t be harvested can’t be decrypted, not now, not ever.
Have a WAN link carrying data that has to stay confidential for years? Visit CyberRidge to see how Carmel fits an existing DWDM network.
FAQs
Q: What is the main difference between Optical encryption, MACsec, and IPsec?
A: The difference is the layer each one works at and the unit of traffic it encrypts. Layer 1 optical encryption encrypts the payload of a transport frame on one link, MACsec protects Ethernet frames between neighboring devices, and IPsec protects IP packets between two addresses. None of them is a replacement for the others.
Q: Is Layer 1 optical encryption faster than MACsec and IPsec?
A: Vendors report that it is. Layer 1 encryption reaches latency in the nanosecond range while MACsec and IPsec show higher latency, and Electronic Design reports below 180 nanoseconds for OTN payload encryption with AES-256.
Q: Does Layer 1 optical encryption stop fiber tapping?
A: No. It encrypts the payload so that a tap cannot read the traffic, but it does not prevent the tap or the copying of the encrypted stream. Detection methods such as OTDR testing look for taps, and Carmel makes a successful tap capture noise.
Q: Does Layer 1 optical encryption protect against harvest now decrypt later attacks?
A: Only in part. AES-256 is regarded as quantum-resistant, and the exposure sits in how the session keys are established. Some vendors have moved that step to post-quantum methods. A recorded copy still exists, and its safety depends on the algorithms holding for the life of the data. Carmel removes the recorded copy from the link it protects.
Q: Does Carmel replace Layer 1 optical encryption from other vendors?
A: Not necessarily. Carmel installs over existing fiber and DWDM equipment and shares the C-band with other wavelengths, so encrypted wavelengths from other vendors can stay in service. Carmel blocks the harvest of the cipher-text, so the encryption is not vulnerable to eventual hacking.