ROOTCON13 CTF – Reverse Engineering – W4RMUP

It’s the time of the year when ROOTCON, the largest security conference in the Philippines, is back in action. This was my 2nd time attending the conference and my 2nd time joining ROOTCON’s Capture the Flag event. Last year’s CTF was a close game since AJ, Ameer, and I placed 2nd from the overall score but we were able to pull off the 1st place this year against other strong teams including our hackstreetboys teammates Ariz, Felix, and Jym from [hsb]JumboHackdog, and IJ from DiKoPoAlam.

It was honestly a challenging game thanks to the awesome guys from Pwn De Manila.

Since AJ and Ameer did the web (nearly a board wipe!) which can be seen here, I was doing other categories and gonna share this reverse engineering challenge that I managed to solve during the event.

This challenge was called “w4rmup”. So initially, after downloading the file, loading it through a hex editor shows that it’s an ELF file:

Checking this through Linux using the file command gives the following output:

Next is running the executable and seeing what it does:

Now to use a debugger such as GDB by typing in:

gdb ./w4rmup

To see functions from the executable, type the command:

info functions

Initially, the function that I investigated was “main”. Doing a “disas main” shows the following output:

0x0000555555554ade <+0>: push   rbp
0x0000555555554adf <+1>: mov    rbp,rsp
0x0000555555554ae2 <+4>: sub    rsp,0x70
0x0000555555554ae6 <+8>: mov    rax,QWORD PTR fs:0x28
0x0000555555554aef <+17>: mov    QWORD PTR [rbp-0x8],rax
0x0000555555554af3 <+21>: xor    eax,eax
0x0000555555554af5 <+23>: mov    BYTE PTR [rbp-0x70],0x31
0x0000555555554af9 <+27>: mov    BYTE PTR [rbp-0x6f],0x7a
0x0000555555554afd <+31>: mov    BYTE PTR [rbp-0x6e],0x5f
0x0000555555554b01 <+35>: mov    BYTE PTR [rbp-0x6d],0x74
0x0000555555554b05 <+39>: mov    BYTE PTR [rbp-0x6c],0x68
0x0000555555554b09 <+43>: mov    BYTE PTR [rbp-0x6b],0x31
0x0000555555554b0d <+47>: mov    BYTE PTR [rbp-0x6a],0x7a
0x0000555555554b11 <+51>: mov    BYTE PTR [rbp-0x69],0x5f
0x0000555555554b15 <+55>: mov    BYTE PTR [rbp-0x68],0x74
0x0000555555554b19 <+59>: mov    BYTE PTR [rbp-0x67],0x68
0x0000555555554b1d <+63>: mov    BYTE PTR [rbp-0x66],0x33
0x0000555555554b21 <+67>: mov    BYTE PTR [rbp-0x65],0x5f
0x0000555555554b25 <+71>: mov    BYTE PTR [rbp-0x64],0x72
0x0000555555554b29 <+75>: mov    BYTE PTR [rbp-0x63],0x33
0x0000555555554b2d <+79>: mov    BYTE PTR [rbp-0x62],0x34
0x0000555555554b31 <+83>: mov    BYTE PTR [rbp-0x61],0x6c
0x0000555555554b35 <+87>: mov    BYTE PTR [rbp-0x60],0x5f
0x0000555555554b39 <+91>: mov    BYTE PTR [rbp-0x5f],0x66
0x0000555555554b3d <+95>: mov    BYTE PTR [rbp-0x5e],0x6c
0x0000555555554b41 <+99>: mov    BYTE PTR [rbp-0x5d],0x34
0x0000555555554b45 <+103>: mov    BYTE PTR [rbp-0x5c],0x67
0x0000555555554b49 <+107>: mov    BYTE PTR [rbp-0x5b],0x5f
0x0000555555554b4d <+111>: mov    BYTE PTR [rbp-0x5a],0x31
0x0000555555554b51 <+115>: mov    BYTE PTR [rbp-0x59],0x7a
0x0000555555554b55 <+119>: mov    BYTE PTR [rbp-0x58],0x5f
0x0000555555554b59 <+123>: mov    BYTE PTR [rbp-0x57],0x74
0x0000555555554b5d <+127>: mov    BYTE PTR [rbp-0x56],0x68
0x0000555555554b61 <+131>: mov    BYTE PTR [rbp-0x55],0x31
0x0000555555554b65 <+135>: mov    BYTE PTR [rbp-0x54],0x7a
0x0000555555554b69 <+139>: mov    BYTE PTR [rbp-0x53],0x5f
0x0000555555554b6d <+143>: mov    BYTE PTR [rbp-0x52],0x6a
0x0000555555554b71 <+147>: mov    BYTE PTR [rbp-0x51],0x75
0x0000555555554b75 <+151>: mov    BYTE PTR [rbp-0x50],0x35
0x0000555555554b79 <+155>: mov    BYTE PTR [rbp-0x4f],0x74
0x0000555555554b7d <+159>: mov    BYTE PTR [rbp-0x4e],0x5f
0x0000555555554b81 <+163>: mov    BYTE PTR [rbp-0x4d],0x66
0x0000555555554b85 <+167>: mov    BYTE PTR [rbp-0x4c],0x34
0x0000555555554b89 <+171>: mov    BYTE PTR [rbp-0x4b],0x6e
0x0000555555554b8d <+175>: mov    BYTE PTR [rbp-0x4a],0x74
0x0000555555554b91 <+179>: mov    BYTE PTR [rbp-0x49],0x34
0x0000555555554b95 <+183>: mov    BYTE PTR [rbp-0x48],0x73
0x0000555555554b99 <+187>: mov    BYTE PTR [rbp-0x47],0x79
0x0000555555554b9d <+191>: mov    BYTE PTR [rbp-0x46],0x0
0x0000555555554ba1 <+195>: mov    ecx,0x0
0x0000555555554ba6 <+200>: mov    edx,0x1
0x0000555555554bab <+205>: mov    esi,0x0
0x0000555555554bb0 <+210>: mov    edi,0x0
0x0000555555554bb5 <+215>: mov    eax,0x0
0x0000555555554bba <+220>: call 0x5555555546c0 <ptrace@plt>
0x0000555555554bbf <+225>: test   rax,rax
0x0000555555554bc2 <+228>: jns 0x555555554bcf <main+241>
0x0000555555554bc4 <+230>: mov    eax,0x0
0x0000555555554bc9 <+235>: mov    DWORD PTR [rax],0x0
0x0000555555554bcf <+241>: lea    rdi,[rip+0x112]        # 0x555555554ce8
0x0000555555554bd6 <+248>: mov    eax,0x0
0x0000555555554bdb <+253>: call 0x5555555546a0 <printf@plt>
0x0000555555554be0 <+258>: mov    rdx,QWORD PTR [rip+0x201429]        # 0x555555756010 <stdin@@GLIBC_2.2.5>
0x0000555555554be7 <+265>: lea    rax,[rbp-0x40]
0x0000555555554beb <+269>: mov    esi,0x2b
0x0000555555554bf0 <+274>: mov    rdi,rax
0x0000555555554bf3 <+277>: call 0x5555555546b0 <fgets@plt>
0x0000555555554bf8 <+282>: test   rax,rax
0x0000555555554bfb <+285>: je  0x555555554c3f <main+353>
0x0000555555554bfd <+287>: lea    rdx,[rbp-0x70]
0x0000555555554c01 <+291>: lea    rax,[rbp-0x40]
0x0000555555554c05 <+295>: mov    rsi,rdx
0x0000555555554c08 <+298>: mov    rdi,rax
0x0000555555554c0b <+301>: call0x5555555547ea <strcmp>
0x0000555555554c10 <+306>: test   al,al
0x0000555555554c12 <+308>: je  0x555555554c22 <main+324>
0x0000555555554c14 <+310>: lea    rdi,[rip+0xe5]        # 0x555555554d00
0x0000555555554c1b <+317>: call0x555555554680 <puts@plt>
0x0000555555554c20 <+322>: jmp 0x555555554c3f <main+353>
0x0000555555554c22 <+324>: lea    rdi,[rip+0xf7]        # 0x555555554d20
0x0000555555554c29 <+331>: mov    eax,0x0
0x0000555555554c2e <+336>: call0x5555555546a0 <printf@plt>
0x0000555555554c33 <+341>: lea    rdi,[rip+0x11d]        # 0x555555554d57
0x0000555555554c3a <+348>: call0x555555554680 <puts@plt>
0x0000555555554c3f <+353>: mov    eax,0x0
0x0000555555554c44 <+358>: mov    rcx,QWORD PTR [rbp-0x8]
0x0000555555554c48 <+362>: xor    rcx,QWORD PTR fs:0x28
0x0000555555554c51 <+371>: je  0x555555554c58 <main+378>
0x0000555555554c53 <+373>: call0x555555554690 <__stack_chk_fail@plt>
0x0000555555554c58 <+378>: leave
0x0000555555554c59 <+379>: ret

Seeing the disassembled code, the instructions from <+23> to <+191> should be interesting:

0x0000555555554af5 <+23>: mov    BYTE PTR [rbp-0x70],0x31
0x0000555555554af9 <+27>: mov    BYTE PTR [rbp-0x6f],0x7a
0x0000555555554afd <+31>: mov    BYTE PTR [rbp-0x6e],0x5f

0x0000555555554b95 <+183>: mov    BYTE PTR [rbp-0x48],0x73
0x0000555555554b99 <+187>: mov    BYTE PTR [rbp-0x47],0x79
0x0000555555554b9d <+191>: mov    BYTE PTR [rbp-0x46],0x0

Noticing that the values of these bytes are from the ASCII range, checking their values in the ASCII table should give us the following equivalent:


To recover this data by taking advantage of GDB, add a breakpoint on <+195> (instruction just right after adding the null byte in memory) by typing the following:

break *0x0000555555554ba1

The command “r” here is a shortcut to run the program. During breakpoint, notice the content:

Going back to the main function, the instructions right after that part calls ptrace which in this case, becomes an anti-debugging code because this can only be used for one process at a time. Since GDB also uses this,  once the code runs, it should break the execution and we won’t be able to continue with the flow:

More information about ptrace as an anti-debugging code can be found here.

One method to bypass this would be patching the ptrace instruction. For the sake of simplicity, the code below will temporarily patch the instructions with “nops” (0x90 in hex). Am mentioning “temporarily” because this will only work during the duration of this debugging session. Permanently patching the instructions can also be done.

set *(unsigned char*)0x0000555555554bba = 0x90
set *(unsigned char*)0x0000555555554bbb = 0x90
set *(unsigned char*)0x0000555555554bbc = 0x90
set *(unsigned char*)0x0000555555554bbd = 0x90
set *(unsigned char*)0x0000555555554bbe = 0x90
set *(unsigned char*)0x0000555555554bbf = 0x90
set *(unsigned char*)0x0000555555554bc0 = 0x90
set *(unsigned char*)0x0000555555554bc1 = 0x90

After these GDB instructions are executed, the result after disassembling the main function again should show:

There are actually other ways to do this too like playing with the flags or modifying the jump instructions. This is just one of many ways to do it.

Now the next instructions could easily be understood:

Prints the text asking to enter the key: 0x0000555555554bdb <+253>:  call   0x5555555546a0 <printf@plt>
Asks for the user input: 0x0000555555554bf3 <+277>:    call   0x5555555546b0 <fgets@plt>

The interesting part here would be from <+287> to <+301>:

If we go back to the main code, rbp-0x70 stores the ASCII text 1z_th1z_th3_r34l_fl4g_1z_th1z_ju5t_f4nt4sy while rbp-0x40 stores our input from fgets and both the strings are passed to the function strcmp.

Should be straightforward right? …… WRONG

Entering the data 1z_th1z_th3_r34l_fl4g_1z_th1z_ju5t_f4nt4sy somehow doesn’t work:

Upon further investigation, the function strcmp had some code which shouldn’t be there. This was the moment that I realized that this was a custom function and not the one from the string library. SNEAKY.

So doing a “disas strcmp” should show us some new bytes from <+35> to <+199> (total of 42 bytes):

0x00005555555547ea <+0>: push   rbp
00005555555547eb <+1>: mov    rbp,rsp
00005555555547ee <+4>: sub    rsp,0x60
00005555555547f2 <+8>: mov    QWORD PTR [rbp-0x58],rdi
00005555555547f6 <+12>: mov    QWORD PTR [rbp-0x60],rsi
00005555555547fa <+16>: mov    rax,QWORD PTR fs:0x28
0000555555554803 <+25>: mov    QWORD PTR [rbp-0x8],rax
0000555555554807 <+29>: xor    eax,eax
0000555555554809 <+31>: mov    BYTE PTR [rbp-0x4d],0x0
000055555555480d <+35>: mov    BYTE PTR [rbp-0x40],0xd1
0000555555554811 <+39>: mov    BYTE PTR [rbp-0x3f],0x9b
0000555555554815 <+43>: mov    BYTE PTR [rbp-0x3e],0xaa
0000555555554819 <+47>: mov    BYTE PTR [rbp-0x3d],0xd
000055555555481d <+51>: mov    BYTE PTR [rbp-0x3c],0x25
0000555555554821 <+55>: mov    BYTE PTR [rbp-0x3b],0x41
0000555555554825 <+59>: mov    BYTE PTR [rbp-0x3a],0xb3
0000555555554829 <+63>: mov    BYTE PTR [rbp-0x39],0xce
000055555555482d <+67>: mov    BYTE PTR [rbp-0x38],0xab
0000555555554831 <+71>: mov    BYTE PTR [rbp-0x37],0x3c
0000555555554835 <+75>: mov    BYTE PTR [rbp-0x36],0xf0
0000555555554839 <+79>: mov    BYTE PTR [rbp-0x35],0xfa
000055555555483d <+83>: mov    BYTE PTR [rbp-0x34],0x88
0000555555554841 <+87>: mov    BYTE PTR [rbp-0x33],0xbb
0000555555554845 <+91>: mov    BYTE PTR [rbp-0x32],0xf8
0000555555554849 <+95>: mov    BYTE PTR [rbp-0x31],0x49
000055555555484d <+99>: mov    BYTE PTR [rbp-0x30],0x61
0000555555554851 <+103>: mov    BYTE PTR [rbp-0x2f],0x70
0000555555554855 <+107>: mov    BYTE PTR [rbp-0x2e],0xc1
0000555555554859 <+111>: mov    BYTE PTR [rbp-0x2d],0x5f
000055555555485d <+115>: mov    BYTE PTR [rbp-0x2c],0x83
0000555555554861 <+119>: mov    BYTE PTR [rbp-0x2b],0xcd
0000555555554865 <+123>: mov    BYTE PTR [rbp-0x2a],0x9b
0000555555554869 <+127>: mov    BYTE PTR [rbp-0x29],0x3f
000055555555486d <+131>: mov    BYTE PTR [rbp-0x28],0xe0
0000555555554871 <+135>: mov    BYTE PTR [rbp-0x27],0xa9
0000555555554875 <+139>: mov    BYTE PTR [rbp-0x26],0x6f
0000555555554879 <+143>: mov    BYTE PTR [rbp-0x25],0xe0
000055555555487d <+147>: mov    BYTE PTR [rbp-0x24],0x1d
0000555555554881 <+151>: mov    BYTE PTR [rbp-0x23],0x37
0000555555554885 <+155>: mov    BYTE PTR [rbp-0x22],0x34
0000555555554889 <+159>: mov    BYTE PTR [rbp-0x21],0xb6
000055555555488d <+163>: mov    BYTE PTR [rbp-0x20],0x8d
0000555555554891 <+167>: mov    BYTE PTR [rbp-0x1f],0xdb
0000555555554895 <+171>: mov    BYTE PTR [rbp-0x1e],0x48
0000555555554899 <+175>: mov    BYTE PTR [rbp-0x1d],0xe2
000055555555489d <+179>: mov    BYTE PTR [rbp-0x1c],0x6d
00005555555548a1 <+183>: mov    BYTE PTR [rbp-0x1b],0x30
00005555555548a5 <+187>: mov    BYTE PTR [rbp-0x1a],0x44
00005555555548a9 <+191>: mov    BYTE PTR [rbp-0x19],0x7
00005555555548ad <+195>: mov    BYTE PTR [rbp-0x18],0x47
00005555555548b1 <+199>: mov    BYTE PTR [rbp-0x17],0xe3
00005555555548b5 <+203>: mov    BYTE PTR [rbp-0x4e],0x69
00005555555548b9 <+207>: mov    DWORD PTR [rbp-0x4c],0x0
00005555555548c0 <+214>: mov    DWORD PTR [rbp-0x48],0x0
00005555555548c7 <+221>: mov    DWORD PTR [rbp-0x44],0x0
00005555555548ce <+228>: mov    DWORD PTR [rbp-0x48],0x0
00005555555548d5 <+235>: jmp 555555554903 <strcmp+281>
00005555555548d7 <+237>: mov    eax, DWORD PTR [rbp-0x48]
00005555555548da <+240>: movsxd rdx,eax
00005555555548dd <+243>: mov    rax,QWORD PTR [rbp-0x58]
00005555555548e1 <+247>: add    rax,rdx
00005555555548e4 <+250>: movzx  edx,BYTE PTR [rax]
00005555555548e7 <+253>: mov    eax,DWORD PTR [rbp-0x48]
00005555555548ea <+256>: movsxd rcx,eax
00005555555548ed <+259>: mov    rax,QWORD PTR [rbp-0x60]
00005555555548f1 <+263>: add    rax,rcx
00005555555548f4 <+266>: movzx  eax,BYTE PTR [rax]
00005555555548f7 <+269>: cmp    dl,al
00005555555548f9 <+271>: jne 5555555548ff <strcmp+277>
00005555555548fb <+273>: add    DWORD PTR [rbp-0x44],0x1
00005555555548ff <+277>: add    DWORD PTR [rbp-0x48],0x1
0000555555554903 <+281>: cmp    DWORD PTR [rbp-0x48],0x29
0000555555554907 <+285>: jle 5555555548d7 <strcmp+237>
0000555555554909 <+287>: cmp    DWORD PTR [rbp-0x44],0x2a
000055555555490d <+291>: jne 555555554918 <strcmp+302>
000055555555490f <+293>: movzx  eax,BYTE PTR [rbp-0x4d]
0000555555554913 <+297>: jmp 5555555549e6 <strcmp+508>
0000555555554918 <+302>: mov    DWORD PTR [rbp-0x4c],0x0
000055555555491f <+309>: jmp 5555555549d4 <strcmp+490>
0000555555554924 <+314>: mov    eax,DWORD PTR [rbp-0x4c]
0000555555554927 <+317>: movsxd rdx,eax
000055555555492a <+320>: mov    rax,QWORD PTR [rbp-0x58]
000055555555492e <+324>: add    rax,rdx
0000555555554931 <+327>: movzx  eax,BYTE PTR [rax]
0000555555554934 <+330>: cmp    BYTE PTR [rbp-0x4e],al
0000555555554937 <+333>: je  55555555495b <strcmp+369>
0000555555554939 <+335>: mov    eax,DWORD PTR [rbp-0x4c]
000055555555493c <+338>: movsxd rdx,eax
000055555555493f <+341>: mov    rax,QWORD PTR [rbp-0x58]
0000555555554943 <+345>: add    rax,rdx
0000555555554946 <+348>: movzx  eax,BYTE PTR [rax]
0000555555554949 <+351>: mov    edx,DWORD PTR [rbp-0x4c]
000055555555494c <+354>: movsxd rcx,edx
000055555555494f <+357>: mov    rdx,QWORD PTR [rbp-0x58]
0000555555554953 <+361>: add    rdx,rcx
0000555555554956 <+364>: xor    al,BYTE PTR [rbp-0x4e]
0000555555554959 <+367>: mov    BYTE PTR [rdx],al
000055555555495b <+369>: mov    eax,DWORD PTR [rbp-0x4c]
000055555555495e <+372>: movsxd rdx,eax
0000555555554961 <+375>: mov    rax,QWORD PTR [rbp-0x58]
0000555555554965 <+379>: add    rax,rdx
0000555555554968 <+382>: movzx  eax,BYTE PTR [rax]
000055555555496b <+385>: movzx  eax,al
000055555555496e <+388>: shl    eax,0x4
0000555555554971 <+391>: mov    ecx,eax
0000555555554973 <+393>: mov    eax,DWORD PTR [rbp-0x4c]
0000555555554976 <+396>: movsxd rdx,eax
0000555555554979 <+399>: mov    rax,QWORD PTR [rbp-0x58]
000055555555497d <+403>: add    rax,rdx
0000555555554980 <+406>: movzx  eax,BYTE PTR [rax]
0000555555554983 <+409>: shr    al,0x4
0000555555554986 <+412>: or     ecx,eax
0000555555554988 <+414>: mov    eax,DWORD PTR [rbp-0x4c]
000055555555498b <+417>: movsxd rdx,eax
000055555555498e <+420>: mov    rax,QWORD PTR [rbp-0x58]
0000555555554992 <+424>: add    rax,rdx
0000555555554995 <+427>: mov    edx,ecx
0000555555554997 <+429>: mov    BYTE PTR [rax],dl
0000555555554999 <+431>: mov    eax,DWORD PTR [rbp-0x4c]
000055555555499c <+434>: movsxd rdx,eax
000055555555499f <+437>: mov    rax,QWORD PTR [rbp-0x58]
00005555555549a3 <+441>: add    rax,rdx
00005555555549a6 <+444>: movzx  eax,BYTE PTR [rax]
00005555555549a9 <+447>: mov    BYTE PTR [rbp-0x4e],al
00005555555549ac <+450>: mov    eax,DWORD PTR [rbp-0x4c]
00005555555549af <+453>: movsxd rdx,eax
00005555555549b2 <+456>: mov    rax,QWORD PTR [rbp-0x58]
00005555555549b6 <+460>: add    rax,rdx
00005555555549b9 <+463>: movzx  edx,BYTE PTR [rax]
00005555555549bc <+466>: mov    eax,DWORD PTR [rbp-0x4c]
00005555555549bf <+469>: cdqe
00005555555549c1 <+471>: movzx  eax,BYTE PTR [rbp+rax*1-0x40]
00005555555549c6 <+476>: cmp    dl,al
00005555555549c8 <+478>: je  5555555549d0 <strcmp+486>
00005555555549ca <+480>: movzx  eax,BYTE PTR [rbp-0x4d]
00005555555549ce <+484>: jmp 5555555549e6 <strcmp+508>
00005555555549d0 <+486>: add    DWORD PTR [rbp-0x4c],0x1
00005555555549d4 <+490>: cmp    DWORD PTR [rbp-0x4c],0x29
00005555555549d8 <+494>: jle 555555554924 <strcmp+314>
00005555555549de <+500>: mov    BYTE PTR [rbp-0x4d],0x1
00005555555549e2 <+504>: movzx  eax,BYTE PTR [rbp-0x4d]
00005555555549e6 <+508>: mov    rsi,QWORD PTR [rbp-0x8]
00005555555549ea <+512>: xor    rsi,QWORD PTR fs:0x28
00005555555549f3 <+521>: je  5555555549fa <strcmp+528>
00005555555549f5 <+523>: call555555554690 <__stack_chk_fail@plt>
00005555555549fa <+528>: leave
00005555555549fb <+529>: ret

This is where the real challenge begins because there are lots of instructions to trace. To break down the code:

  1. After the seemingly garbage bytes are placed in memory, the code jumps to <strcmp+281> where a comparison of a counter is done with 0x29 (41 in decimal). If the counter is less than or equal to 0x29, function jumps to <strcmp+237>.
  2. Instructions from <+237> to <+269> try to compare the user’s input data with 1z_th1z_th3_r34l_fl4g_1z_th1z_ju5t_f4nt4sy and increments a counter with each correct match. If all characters match, the code will jump to <strcmp+508> which exits the function as seen from <+297>:


0000555555554913 <+297>: jmp 5555555549e6 <strcmp+508>

This basically means that the string 1z_th1z_th3_r34l_fl4g_1z_th1z_ju5t_f4nt4sy is not the correct flag (It’s just fantasy).

  1. If the input data doesn’t match the string 1z_th1z_th3_r34l_fl4g_1z_th1z_ju5t_f4nt4sy the code will jump to <strcmp+302> and starts looping through the input string character by character.
  2. The first iteration of the loop grabs the value from rbp-0x4e which is initially 0x69 and tries to compare it with the first character of the input. If it matches, code jumps to <strcmp+369> but if it doesn’t match, both get XOR’d.
  3. Whether the value was XOR’d or not, the next step is shifting the character to the left by 4 bits as seen in <+388>:

shl eax, 0x4

After shifting to the left, the result gets stored in ECX and the original character is also shifted to the right by 4 bits as seen in <+409>:

shr al, 0x4

  1. Both results are then OR’d and the final result gets stored in [rbp-0x4e] where it had the initial value of 0x69.
  2. The result from the Bitwise OR operation is then compared to the garbage data from the start of the strcmp function.

The algorithm follows as:

  • Compare input with 1z_th1z_th3_r34l_fl4g_1z_th1z_ju5t_f4nt4sy
    • If input matches, exit the function
    • If input doesn’t match continue to the next step
  • Iterate through each character of the input
    • Suppose x = 0x69, counter = 0x00
    • If input[counter] != x:
      • input[counter] = input[counter] xor x
    • A = input[counter] << 4
    • B = input[counter] >> 4
    • C = [rbp-0x4e] = A OR B
    • Check garbage_values[counter] == C

The problem however is that this algorithm isn’t reversible because of the bitwise OR operation. The reason for this is because of OR’s truth table where if any input bit is ‘1’, the result will always be ‘1’ whether the other value is a ‘0’ or ‘1’.

So if we had a hex number of 0xC1 and we use the bitwise operator OR with 0xC0, the result will be 0xC1 and there’s no exact way to tell how that 0xC1 was derived because some other values that can present the exact result are:

0xC1 OR 0x01 = 0xC1
0xC1 OR 0x41 = 0xC1
0xC1 OR 0x81 = 0xC1

This means that we must test all the possible values in the range to see what matches the result (C# program):

static void Main(string[] args)
    byte[] garbage = new byte[] { 0xd1, 0x9b, 0xaa, 0xd, 0x25, 0x41, 0xb3, 0xce, 0xab, 0x3c, 0xf0,
    0xfa, 0x88, 0xbb, 0xf8, 0x49, 0x61, 0x70, 0xc1, 0x5f, 0x83, 0xcd, 0x9b, 0x3f, 0xe0, 0xa9, 0x6f,
    0xe0, 0x1d, 0x37, 0x34, 0xb6, 0x8d, 0xdb, 0x48, 0xe2, 0x6d, 0x30, 0x44, 0x07, 0x47, 0xe3 };

    int counter = 0;
    int rbp_4e = 0x69;

    // Loop through all garbage chars
    while (counter < garbage.Length)
        // Test values from 0-255
        for (int possible_input = 0; possible_input < 256; possible_input++)
            int i = possible_input;
            if (possible_input != rbp_4e)
                i = i ^ rbp_4e;
            int c = (i << 4 | i >> 4) % 256;

            if (garbage[counter] == c)
                rbp_4e = c;

Flag: th1z_1z_th3_r34l_fl4g_th1z_a1nt_n0_f4nt4sy

Overall, I found this quite challenging but it definitely feels good to finally grab that flag after hours of working on this. Here’s a picture of ROOTCON’s Black Badge:

Shout out to my ROOTCON friends, hackstreetboys teammates, and PWN De Manila!

Leave a Reply