Avoid signed overflow in bs_read_se() (#2071)

bs_read_ue() can return values above INT32_MAX. bs_read_se() stored the
result in an int32_t and computed (r+1)/2, which is undefined behavior
when the code is 0x7FFFFFFF and gives wrong values for codes above
INT32_MAX. The SPS being parsed comes from the host during the H.264
SPS fixup.

Do the conversion in unsigned arithmetic. bs_read_ue() returns at most
0xFFFFFFFE, so both branches produce in-range int32_t values, and every
code that was previously well-defined decodes to the same value.
This commit is contained in:
Chris authored and GitHub committed 2026-10-24 18:17:00 -05:00
1 parent 781b26fd1c
commit c8474c515c
1 file changed
+6 -4
+6 -4
View File
@@ -178,16 +178,18 @@ static inline uint32_t bs_read_ue(bs_t* b)
static inline int32_t bs_read_se(bs_t* b)
{
int32_t r = bs_read_ue(b);
// Do the math in unsigned space. bs_read_ue() can return values above
// INT32_MAX, and (r+1)/2 on a signed INT32_MAX is undefined behavior.
// bs_read_ue() returns at most 0xFFFFFFFE, so both results fit in int32_t.
uint32_t r = bs_read_ue(b);
if (r & 0x01)
{
r = (r+1)/2;
return (int32_t)((r >> 1) + 1);
}
else
{
r = -(r/2);
return -(int32_t)(r >> 1);
}
return r;
}
static inline void bs_write_u1(bs_t* b, uint32_t v)