Source code and results are in the attached zip.
My OpenSSL encrypts files converted to Base64 so that we only deal with text files. The resul is also a PEM text file (I prefer text files).
openssl cms -encrypt -in test.zip.b64 -aes256 -rand {noiseFile} -out {Globals.outFolder}\test_mickey.zip.b64.enc -recip mickey_cert.pem -keyopt ecdh_kdf_md:sha256 I am using the patest 2.6.2 release downloaded from nuget: BouncyCastle.Cryptography 2.6.2 nuget
Here is my call to Bounty Castle which created a binary DER file:
msEcdhInteropTest.EncryptFile(b64OutBC, $"{b64OutBC}.BCenc", receiverCertBC)
myfile.txt.b64.zip
Analysis of the attached results:
OpenSSL: (11 bytes total — just the OID, no parameters field)
parameter: SEQUENCE:
0:d=0 hl=2 l= 11 cons: SEQUENCE
2:d=1 hl=2 l= 9 prim: OBJECT :id-aes256-wrap
Bouny Castle:
parameter: SEQUENCE: (13 bytes — OID plus an explicit NULL parameters field)
0:d=0 hl=2 l= 13 cons: SEQUENCE
2:d=1 hl=2 l= 9 prim: OBJECT :id-aes256-wrap
13:d=1 hl=2 l= 0 prim: NULL
Claude.ai analysis:
This is the exact discrepancy that breaks interop. That inner wrap-algorithm AlgorithmIdentifier gets DER-encoded into the ECC-CMS-SharedInfo structure (RFC 5753) that's fed into the X9.63 KDF alongside the raw ECDH secret Z. Two extra bytes there means different KDF input, means a different derived key-encryption-key — everything else lines up (curve, KDF OID, wrap OID, content cipher), but the actual key bytes silently diverge. That produces exactly the deterministic "checksum failed" you're seeing, with no other symptom, because the algorithms genuinely do all match — only the encoding of one of them differs.
This is a known category of interop bug: some ASN.1 libraries include an explicit NULL for "no parameters" (strict X.690 tag-length-value completeness), others correctly follow RFC 5753's convention of omitting the parameters field entirely for this specific algorithm. OpenSSL follows the RFC. BC 2.6.2, apparently, does not — matching what you were "hinted" about their ASN.1 not being bug-free.
This means BC's high-level AddKeyAgreementRecipient API can't produce openssl-compatible output as-is, and likely can't be worked around by changing OIDs or wrap choices — the NULL insertion is happening inside BC's internal SharedInfo/AlgorithmIdentifier construction, not something the public API surface lets you override.
Please check.
Source code and results are in the attached zip.
My OpenSSL encrypts files converted to Base64 so that we only deal with text files. The resul is also a PEM text file (I prefer text files).
openssl cms -encrypt -in test.zip.b64 -aes256 -rand {noiseFile} -out {Globals.outFolder}\test_mickey.zip.b64.enc -recip mickey_cert.pem -keyopt ecdh_kdf_md:sha256I am using the patest 2.6.2 release downloaded from nuget: BouncyCastle.Cryptography 2.6.2 nugetHere is my call to Bounty Castle which created a binary DER file:
msEcdhInteropTest.EncryptFile(b64OutBC, $"{b64OutBC}.BCenc", receiverCertBC)myfile.txt.b64.zip
Analysis of the attached results:
OpenSSL: (11 bytes total — just the OID, no parameters field)
Bouny Castle:
parameter: SEQUENCE: (13 bytes — OID plus an explicit NULL parameters field)
Claude.ai analysis:
This is the exact discrepancy that breaks interop. That inner wrap-algorithm AlgorithmIdentifier gets DER-encoded into the ECC-CMS-SharedInfo structure (RFC 5753) that's fed into the X9.63 KDF alongside the raw ECDH secret Z. Two extra bytes there means different KDF input, means a different derived key-encryption-key — everything else lines up (curve, KDF OID, wrap OID, content cipher), but the actual key bytes silently diverge. That produces exactly the deterministic "checksum failed" you're seeing, with no other symptom, because the algorithms genuinely do all match — only the encoding of one of them differs.
This is a known category of interop bug: some ASN.1 libraries include an explicit NULL for "no parameters" (strict X.690 tag-length-value completeness), others correctly follow RFC 5753's convention of omitting the parameters field entirely for this specific algorithm. OpenSSL follows the RFC. BC 2.6.2, apparently, does not — matching what you were "hinted" about their ASN.1 not being bug-free.
This means BC's high-level AddKeyAgreementRecipient API can't produce openssl-compatible output as-is, and likely can't be worked around by changing OIDs or wrap choices — the NULL insertion is happening inside BC's internal SharedInfo/AlgorithmIdentifier construction, not something the public API surface lets you override.
Please check.