Problem was, that addiu is "Add immediate unsigned", but immediate is still taken as sign-extended value, so the sign-bit is most likely ignored when adding the number. I changed the code to use "addu" and it works
.
This is the new disassembled code:
Dump of assembler code from 0x2ab68964 to 0x2ab68984 (malloc):
0x2ab68964: lui t9,0x3000
0x2ab68968: li at,0x0
0x2ab6896c: addu t9,t9,at
0x2ab68970: jr t9
0x2ab68974: nop
0x2ab68978: sw s8,80(sp)
0x2ab6897c: sw s7,76(sp)
0x2ab68980: sw s6,72(sp)
End of assembler dump.
(gdb) disassemble 0x30000000, 0x30000038
Dump of assembler code from 0x30000000 to 0x30000038:
; reset t9 to value it would have in original malloc
0x30000000: lui t9,0x2ab6
0x30000004: li at,0x8964
0x30000008: addu t9,t9,at
; exec overwritten instructions from original malloc
0x3000000c: lui gp,0x6
0x30000010: addiu gp,gp,-25780
0x30000014: addu gp,gp,t9
0x30000018: addiu sp,sp,-88
0x3000001c: sw ra,84(sp)
; jump back to original malloc+20
0x30000020: lui t9,0x2ab6
0x30000024: li at,0x8978
0x30000028: addu t9,t9,at
0x3000002c: jr t9
0x30000030: nop