Integer Overflow at t1.c:1989

[headshog]

Hi! We've been fuzzing openjpeg with sydr-fuzz security predicates and we found integer overflow error in t1.c:1989.

In function opj_t1_decode_cblk at line 1989 integer overflow occurs in variable bpno_plus_one (cblk->numbps was too big (4294967288)). Then bpno_plus_one is checked for if (bpno_plus_one >= 31), so after overflow the result of this if condition may be incorrect.

Environment

• OS: ubuntu 20.04
• commit: 6af3931

How to reproduce this error

1. Build docker container:

   sudo docker build -t oss-sydr-fuzz-openjpeg .
   

2. Run docker container:

   sudo docker run --privileged --network host -v /etc/localtime:/etc/localtime:ro --rm -it -v $PWD:/fuzz oss-sydr-fuzz-openjpeg /bin/bash
   

3. Run on the following input:

    /opj_decompress_fuzzer_J2K_fuzz sydr_t1.txt
   

4. Output:

   /openjpeg/src/lib/openjp2/t1.c:1989:42: runtime error: unsigned integer overflow: 9 + 4294967288 cannot be represented in type 'unsigned int'
   SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior /openjpeg/src/lib/openjp2/t1.c:1989:42
   

[rouault]

this overflow can't occur once the fix of #1488 has been applied