Fix buffer under/overflows in `SmokeLine` (#132)

When `SmokeLine` is called through `DrawTheGlow`, the calculated shade
table offset might result in a negative value. Subject to usual
arithmetic conversions, it is then treated as a large unsigned array
index. On 32-bit systems (e.g. OG builds for DOS, Windows 95), the
pointer arithmetic overflows and produces a negative index access,
simply grabbing game data a few bytes before the start of the table. On
64-bit platforms this instead results in page fault.

1. Keep table offset in a signed integer, making the negative values
   explicit. This provides OG behavior for 64-bit builds.
2. Optionally, cap all negative values at 0, preventing underflows.

Signed-off-by: Artur Rojek <contact@artur-rojek.eu>
This commit is contained in:
Artur Rojek 2023-02-26 12:55:20 +01:00
parent 48b70bb8ff
commit 12e941081a
1 changed files with 8 additions and 1 deletions

View File

@ -858,6 +858,7 @@ void DrMatrix34Rotate(br_matrix34* mat, br_angle r, br_vector3* a) {
// IDA: void __usercall SmokeLine(int l@<EAX>, int x@<EDX>, br_scalar zbuff, int r_squared, tU8 *scr_ptr, tU16 *depth_ptr, tU8 *shade_ptr, br_scalar r_multiplier, br_scalar z_multiplier, br_scalar shade_offset)
void SmokeLine(int l, int x, br_scalar zbuff, int r_squared, tU8* scr_ptr, tU16* depth_ptr, tU8* shade_ptr, br_scalar r_multiplier, br_scalar z_multiplier, br_scalar shade_offset) {
int i;
int offset; /* Added by dethrace. */
int r_multiplier_int;
int shade_offset_int;
tU16 z;
@ -873,7 +874,13 @@ void SmokeLine(int l, int x, br_scalar zbuff, int r_squared, tU8* scr_ptr, tU16*
for (i = 0; i < l; i++) {
if (*depth_ptr > z) {
*scr_ptr = shade_ptr[*scr_ptr + (((shade_offset_int - r_squared * r_multiplier_int) >> 8) & 0xffffff00)];
offset = ((shade_offset_int - r_squared * r_multiplier_int) >> 8) &
0xffffff00;
#if defined(DETHRACE_FIX_BUGS)
/* Prevent buffer underflows by capping negative offsets. */
offset = MAX(0, offset);
#endif
*scr_ptr = shade_ptr[*scr_ptr + offset];
}
r_multiplier = x + r_squared;
scr_ptr++;