Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

std.crypto: Add secure DER parser #19775

Open
clickingbuttons opened this issue Apr 26, 2024 · 0 comments · May be fixed by #19976
Open

std.crypto: Add secure DER parser #19775

clickingbuttons opened this issue Apr 26, 2024 · 0 comments · May be fixed by #19976
Labels
standard library This issue involves writing Zig code for the standard library.
Milestone
0.14.0

Comments

@clickingbuttons
Copy link
Contributor

The current DER parser lacks:

  • Checks to prevent out-of-bounds reads by verifying slice lengths are indeed inside its byte slice.
  • Checks to prevent hash collision attacks by verifying fields are in their shortest canonical form.
  • Checks to prevent hash collision attacks by verifying no bytes are unparsed.
  • OID parsing for auditing (see audit the standard library TLS implementation with respect to RFC 8446 #14178).
  • Tests.
  • An API to easily expect an element.

zig/lib/std/crypto/Certificate.zig

Lines 864 to 946 in 8af59d1

pub const der = struct {
pub const Class = enum(u2) {
universal,
application,
context_specific,
private,
};
pub const PC = enum(u1) {
primitive,
constructed,
};
pub const Identifier = packed struct(u8) {
tag: Tag,
pc: PC,
class: Class,
};
pub const Tag = enum(u5) {
boolean = 1,
integer = 2,
bitstring = 3,
octetstring = 4,
null = 5,
object_identifier = 6,
sequence = 16,
sequence_of = 17,
utc_time = 23,
generalized_time = 24,
_,
};
pub const Element = struct {
identifier: Identifier,
slice: Slice,
pub const Slice = struct {
start: u32,
end: u32,
pub const empty: Slice = .{ .start = 0, .end = 0 };
};
pub const ParseElementError = error{CertificateFieldHasInvalidLength};
pub fn parse(bytes: []const u8, index: u32) ParseElementError!Element {
var i = index;
const identifier = @as(Identifier, @bitCast(bytes[i]));
i += 1;
const size_byte = bytes[i];
i += 1;
if ((size_byte >> 7) == 0) {
return .{
.identifier = identifier,
.slice = .{
.start = i,
.end = i + size_byte,
},
};
}
const len_size = @as(u7, @truncate(size_byte));
if (len_size > @sizeOf(u32)) {
return error.CertificateFieldHasInvalidLength;
}
const end_i = i + len_size;
var long_form_size: u32 = 0;
while (i < end_i) : (i += 1) {
long_form_size = (long_form_size << 8) | bytes[i];
}
return .{
.identifier = identifier,
.slice = .{
.start = i,
.end = i + long_form_size,
},
};
}
};
};

@clickingbuttons clickingbuttons mentioned this issue Apr 26, 2024
Closed
@clickingbuttons clickingbuttons changed the title Add secure DER parser std.crypto: Add secure DER parser Apr 27, 2024
clickingbuttons added a commit to clickingbuttons/zig that referenced this issue May 15, 2024
@clickingbuttons
std.crypto: Add ASN1 module with OIDs and DER
760d742 
Add module for mapping ASN1 types to Zig types. See `asn1.Tag.fromZig` for the
mapping. Add DER encoder and decoder.

ASN1 is an abstract encoding scheme generally formatted like:
1. Tag
	- Number: An identifier. I've chosen a reasonable u16 limit.
	- Constructed: Whether this type is a container.
	- Class: Number's namespace.
2. Length of following value
3. Value

This implementation allows ASN1 tags to be overriden with `asn1_tag`
and `asn1_tags`:
```zig
const MyContainer = (enum | union | struct) {
	pub const asn1_tag = asn1.Tag.init(...);

	// This specifies a tag's class, and if explicit, additional encoding
	// rules.
	pub const asn1_tags = .{
		.field = asn1.FieldTag.explicit(0, .context_specific),
	};
};
```

Despite having an enumeration type, ASN1 frequently uses OIDs as enum
values. This is supported via an `pub const oids` field.
```zig
const MyEnum = enum {
	a,

	pub const oids = asn1.Oid.StaticMap(MyEnum).initComptime(.{
		.a = "1.2.3.4",
	});
};
```

Futhermore, a container may choose to implement encoding and decoding
however it deems fit. This allows for derived fields since Zig has a far
more powerful type system than ASN1.
```zig
// ASN1 has no standard way of tagging unions.
const MyContainer = union(enum) {
	derived: PowerfulZigType,

	const WeakAsn1Type = ...;

	pub fn encodeDer(self: MyContainer, encoder: *der.Encoder) !void {
		try encoder.any(WeakAsn1Type{...});
	}

	pub fn decodeDer(decoder: *der.Decoder) !MyContainer {
		const weak_asn1_type = try decoder.any(WeakAsn1Type);
		return .{ .derived = PowerfulZigType{...} };
	}
};
```
This should prevent having to reparse ASN1 structures into workable Zig
data structures like in certificate parsing. An unfortunate side-effect
is that decoding and encoding cannot have complete error sets unless we
limit what errors users may return. Luckily, PKI ASN1 types are NOT
recursive so the inferred error set should be sufficient.

Finally, other encodings are possible, but this patch only implements DER.

This PR does not actually use the parser for stdlib PKI, but an example of
how Certificate may be refactored is available
[here.](https://github.com/clickingbuttons/asn1/blob/69c5709d/src/Certificate.zig)

Closes ziglang#19775.
clickingbuttons added a commit to clickingbuttons/zig that referenced this issue May 15, 2024
@clickingbuttons
std.crypto: Add ASN1 module with OIDs and DER
df2bb8e 
Add module for mapping ASN1 types to Zig types. See `asn1.Tag.fromZig` for the
mapping. Add DER encoder and decoder.

ASN1 is an abstract encoding scheme generally formatted like:
1. Tag
	- Number: An identifier. I've chosen a reasonable u16 limit.
	- Constructed: Whether this type is a container.
	- Class: Number's namespace.
2. Length of following value
3. Value

This implementation allows ASN1 tags to be overriden with `asn1_tag`
and `asn1_tags`:
```zig
const MyContainer = (enum | union | struct) {
	field: u32,

	pub const asn1_tag = asn1.Tag.init(...);

	// This specifies a tag's class, and if explicit, additional encoding
	// rules.
	pub const asn1_tags = .{
		.field = asn1.FieldTag.explicit(0, .context_specific),
	};
};
```

Despite having an enum tag type, ASN1 frequently uses OIDs as enum
values. This is supported via an `pub const oids` field.
```zig
const MyEnum = enum {
	a,

	pub const oids = asn1.Oid.StaticMap(MyEnum).initComptime(.{
		.a = "1.2.3.4",
	});
};
```

Futhermore, a container may choose to implement encoding and decoding
however it deems fit. This allows for derived fields since Zig has a far
more powerful type system than ASN1.
```zig
// ASN1 has no standard way of tagging unions.
const MyContainer = union(enum) {
	derived: PowerfulZigType,

	const WeakAsn1Type = ...;

	pub fn encodeDer(self: MyContainer, encoder: *der.Encoder) !void {
		try encoder.any(WeakAsn1Type{...});
	}

	pub fn decodeDer(decoder: *der.Decoder) !MyContainer {
		const weak_asn1_type = try decoder.any(WeakAsn1Type);
		return .{ .derived = PowerfulZigType{...} };
	}
};
```
An unfortunate side-effect is that decoding and encoding cannot have
complete complete error sets unless we limit what errors users may
return. Luckily, PKI ASN1 types are NOT recursive so the inferred
error set should be sufficient.

Finally, other encodings are possible, but this patch only implements
a buffered DER encoder and decoder.

In an effort to keep the changeset minimal this PR does not actually
use the DER parser for stdlib PKI, but an example of how it may be
used for Certificate is available
[here.](https://github.com/clickingbuttons/asn1/blob/69c5709d/src/Certificate.zig)

Closes ziglang#19775.
clickingbuttons added a commit to clickingbuttons/zig that referenced this issue May 15, 2024
@clickingbuttons
std.crypto: Add ASN1 module with OIDs and DER
6dbb679 
Add module for mapping ASN1 types to Zig types. See `asn1.Tag.fromZig` for the
mapping. Add DER encoder and decoder.

ASN1 is an abstract encoding scheme generally formatted like:
1. Tag
	- Number: An identifier. I've chosen a reasonable u16 limit.
	- Constructed: Whether this type is a container.
	- Class: Number's namespace.
2. Length of following value
3. Value

This implementation allows ASN1 tags to be overriden with `asn1_tag`
and `asn1_tags`:
```zig
const MyContainer = (enum | union | struct) {
	field: u32,

	pub const asn1_tag = asn1.Tag.init(...);

	// This specifies a tag's class, and if explicit, additional encoding
	// rules.
	pub const asn1_tags = .{
		.field = asn1.FieldTag.explicit(0, .context_specific),
	};
};
```

Despite having an enum tag type, ASN1 frequently uses OIDs as enum
values. This is supported via an `pub const oids` field.
```zig
const MyEnum = enum {
	a,

	pub const oids = asn1.Oid.StaticMap(MyEnum).initComptime(.{
		.a = "1.2.3.4",
	});
};
```

Futhermore, a container may choose to implement encoding and decoding
however it deems fit. This allows for derived fields since Zig has a far
more powerful type system than ASN1.
```zig
// ASN1 has no standard way of tagging unions.
const MyContainer = union(enum) {
	derived: PowerfulZigType,

	const WeakAsn1Type = ...;

	pub fn encodeDer(self: MyContainer, encoder: *der.Encoder) !void {
		try encoder.any(WeakAsn1Type{...});
	}

	pub fn decodeDer(decoder: *der.Decoder) !MyContainer {
		const weak_asn1_type = try decoder.any(WeakAsn1Type);
		return .{ .derived = PowerfulZigType{...} };
	}
};
```
An unfortunate side-effect is that decoding and encoding cannot have
complete complete error sets unless we limit what errors users may
return. Luckily, PKI ASN1 types are NOT recursive so the inferred
error set should be sufficient.

Finally, other encodings are possible, but this patch only implements
a buffered DER encoder and decoder.

In an effort to keep the changeset minimal this PR does not actually
use the DER parser for stdlib PKI, but an example of how it may be
used for Certificate is available
[here.](https://github.com/clickingbuttons/asn1/blob/69c5709d/src/Certificate.zig)

Closes ziglang#19775.
clickingbuttons added a commit to clickingbuttons/zig that referenced this issue May 15, 2024
@clickingbuttons
std.crypto: Add ASN1 module with OIDs and DER
3b34d4c 
Add module for mapping ASN1 types to Zig types. See `asn1.Tag.fromZig` for the
mapping. Add DER encoder and decoder.

ASN1 is an abstract encoding scheme generally formatted like:
1. Tag
    - Number: An identifier. I've chosen a reasonable u16 limit.
    - Constructed: Whether this type is a container.
    - Class: Number's namespace.
2. Length of following value
3. Value

This implementation allows ASN1 tags to be overriden with `asn1_tag`
and `asn1_tags`:
```zig
const MyContainer = (enum | union | struct) {
    field: u32,

    pub const asn1_tag = asn1.Tag.init(...);

    // This specifies a tag's class, and if explicit, additional encoding
    // rules.
    pub const asn1_tags = .{
        .field = asn1.FieldTag.explicit(0, .context_specific),
    };
};
```

Despite having an enum tag type, ASN1 frequently uses OIDs as enum
values. This is supported via an `pub const oids` field.
```zig
const MyEnum = enum {
    a,

    pub const oids = asn1.Oid.StaticMap(MyEnum).initComptime(.{
        .a = "1.2.3.4",
    });
};
```

Futhermore, a container may choose to implement encoding and decoding
however it deems fit. This allows for derived fields since Zig has a far
more powerful type system than ASN1.
```zig
// ASN1 has no standard way of tagging unions.
const MyContainer = union(enum) {
    derived: PowerfulZigType,

    const WeakAsn1Type = ...;

    pub fn encodeDer(self: MyContainer, encoder: *der.Encoder) !void {
        try encoder.any(WeakAsn1Type{...});
    }

    pub fn decodeDer(decoder: *der.Decoder) !MyContainer {
        const weak_asn1_type = try decoder.any(WeakAsn1Type);
        return .{ .derived = PowerfulZigType{...} };
    }
};
```
An unfortunate side-effect is that decoding and encoding cannot have
complete complete error sets unless we limit what errors users may
return. Luckily, PKI ASN1 types are NOT recursive so the inferred
error set should be sufficient.

Finally, other encodings are possible, but this patch only implements
a buffered DER encoder and decoder.

In an effort to keep the changeset minimal this PR does not actually
use the DER parser for stdlib PKI, but an example of how it may be
used for Certificate is available
[here.](https://github.com/clickingbuttons/asn1/blob/69c5709d/src/Certificate.zig)

Closes ziglang#19775.
clickingbuttons added a commit to clickingbuttons/zig that referenced this issue May 15, 2024
@clickingbuttons
std.crypto: Add ASN1 module with OIDs and DER
5ddb5be 
Add module for mapping ASN1 types to Zig types. See
`asn1.Tag.fromZig` for the mapping. Add DER encoder and decoder.

ASN1 is an abstract encoding scheme generally formatted like:
1. Tag
    - Number: An identifier. I've chosen a reasonable u16 limit.
    - Constructed: Whether this type is a container.
    - Class: Number's namespace.
2. Length of following value
3. Value

This implementation allows ASN1 tags to be overriden with `asn1_tag`
and `asn1_tags`:
```zig
const MyContainer = (enum | union | struct) {
    field: u32,

    pub const asn1_tag = asn1.Tag.init(...);

    // This specifies a tag's class, and if explicit, additional encoding
    // rules.
    pub const asn1_tags = .{
        .field = asn1.FieldTag.explicit(0, .context_specific),
    };
};
```

Despite having an enum tag type, ASN1 frequently uses OIDs as enum
values. This is supported via an `pub const oids` field.
```zig
const MyEnum = enum {
    a,

    pub const oids = asn1.Oid.StaticMap(MyEnum).initComptime(.{
        .a = "1.2.3.4",
    });
};
```

Futhermore, a container may choose to implement encoding and decoding
however it deems fit. This allows for derived fields since Zig has a far
more powerful type system than ASN1.
```zig
// ASN1 has no standard way of tagging unions.
const MyContainer = union(enum) {
    derived: PowerfulZigType,

    const WeakAsn1Type = ...;

    pub fn encodeDer(self: MyContainer, encoder: *der.Encoder) !void {
        try encoder.any(WeakAsn1Type{...});
    }

    pub fn decodeDer(decoder: *der.Decoder) !MyContainer {
        const weak_asn1_type = try decoder.any(WeakAsn1Type);
        return .{ .derived = PowerfulZigType{...} };
    }
};
```
An unfortunate side-effect is that decoding and encoding cannot have
complete complete error sets unless we limit what errors users may
return. Luckily, PKI ASN1 types are NOT recursive so the inferred
error set should be sufficient.

Finally, other encodings are possible, but this patch only implements
a buffered DER encoder and decoder.

In an effort to keep the changeset minimal this PR does not actually
use the DER parser for stdlib PKI, but an example of how it may be
used for Certificate is available
[here.](https://github.com/clickingbuttons/asn1/blob/69c5709d/src/Certificate.zig)

Closes ziglang#19775.
clickingbuttons added a commit to clickingbuttons/zig that referenced this issue May 15, 2024
@clickingbuttons
std.crypto: Add ASN1 module with OIDs and DER
3c36279 
Add module for mapping ASN1 types to Zig types. See
`asn1.Tag.fromZig` for the mapping. Add DER encoder and decoder.

See `asn1/test.zig` for example usage of every ASN1 type.

This implementation allows ASN1 tags to be overriden with `asn1_tag`
and `asn1_tags`:
```zig
const MyContainer = (enum | union | struct) {
    field: u32,

    pub const asn1_tag = asn1.Tag.init(...);

    // This specifies a tag's class, and if explicit, additional encoding
    // rules.
    pub const asn1_tags = .{
        .field = asn1.FieldTag.explicit(0, .context_specific),
    };
};
```

Despite having an enum tag type, ASN1 frequently uses OIDs as enum
values. This is supported via an `pub const oids` field.
```zig
const MyEnum = enum {
    a,

    pub const oids = asn1.Oid.StaticMap(MyEnum).initComptime(.{
        .a = "1.2.3.4",
    });
};
```

Futhermore, a container may choose to implement encoding and decoding
however it deems fit. This allows for derived fields since Zig has a far
more powerful type system than ASN1.
```zig
// ASN1 has no standard way of tagging unions.
const MyContainer = union(enum) {
    derived: PowerfulZigType,

    const WeakAsn1Type = ...;

    pub fn encodeDer(self: MyContainer, encoder: *der.Encoder) !void {
        try encoder.any(WeakAsn1Type{...});
    }

    pub fn decodeDer(decoder: *der.Decoder) !MyContainer {
        const weak_asn1_type = try decoder.any(WeakAsn1Type);
        return .{ .derived = PowerfulZigType{...} };
    }
};
```
An unfortunate side-effect is that decoding and encoding cannot have
complete complete error sets unless we limit what errors users may
return. Luckily, PKI ASN1 types are NOT recursive so the inferred
error set should be sufficient.

Finally, other encodings are possible, but this patch only implements
a buffered DER encoder and decoder.

In an effort to keep the changeset minimal this PR does not actually
use the DER parser for stdlib PKI, but a tested example of how it may
be used for Certificate is available
[here.](https://github.com/clickingbuttons/asn1/blob/69c5709d/src/Certificate.zig)

Closes ziglang#19775.
@clickingbuttons clickingbuttons linked a pull request May 15, 2024 that will close this issue
Open
clickingbuttons added a commit to clickingbuttons/zig that referenced this issue May 15, 2024
@clickingbuttons
std.crypto: Add ASN1 module with OIDs and DER
330d353 
Add module for mapping ASN1 types to Zig types. See
`asn1.Tag.fromZig` for the mapping. Add DER encoder and decoder.

See `asn1/test.zig` for example usage of every ASN1 type.

This implementation allows ASN1 tags to be overriden with `asn1_tag`
and `asn1_tags`:
```zig
const MyContainer = (enum | union | struct) {
    field: u32,

    pub const asn1_tag = asn1.Tag.init(...);

    // This specifies a tag's class, and if explicit, additional encoding
    // rules.
    pub const asn1_tags = .{
        .field = asn1.FieldTag.explicit(0, .context_specific),
    };
};
```

Despite having an enum tag type, ASN1 frequently uses OIDs as enum
values. This is supported via an `pub const oids` field.
```zig
const MyEnum = enum {
    a,

    pub const oids = asn1.Oid.StaticMap(MyEnum).initComptime(.{
        .a = "1.2.3.4",
    });
};
```

Futhermore, a container may choose to implement encoding and decoding
however it deems fit. This allows for derived fields since Zig has a far
more powerful type system than ASN1.
```zig
// ASN1 has no standard way of tagging unions.
const MyContainer = union(enum) {
    derived: PowerfulZigType,

    const WeakAsn1Type = ...;

    pub fn encodeDer(self: MyContainer, encoder: *der.Encoder) !void {
        try encoder.any(WeakAsn1Type{...});
    }

    pub fn decodeDer(decoder: *der.Decoder) !MyContainer {
        const weak_asn1_type = try decoder.any(WeakAsn1Type);
        return .{ .derived = PowerfulZigType{...} };
    }
};
```
An unfortunate side-effect is that decoding and encoding cannot have
complete complete error sets unless we limit what errors users may
return. Luckily, PKI ASN1 types are NOT recursive so the inferred
error set should be sufficient.

Finally, other encodings are possible, but this patch only implements
a buffered DER encoder and decoder.

In an effort to keep the changeset minimal this PR does not actually
use the DER parser for stdlib PKI, but a tested example of how it may
be used for Certificate is available
[here.](https://github.com/clickingbuttons/asn1/blob/69c5709d/src/Certificate.zig)

Closes ziglang#19775.
@Vexu Vexu added the standard library This issue involves writing Zig code for the standard library. label May 31, 2024
@Vexu Vexu added this to the 0.14.0 milestone May 31, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
standard library This issue involves writing Zig code for the standard library.
Projects
None yet
Milestone
0.14.0
Development

Successfully merging a pull request may close this issue.

std.crypto: Add ASN1 module with OIDs and DER clickingbuttons/zig
2 participants
@Vexu @clickingbuttons