Upon running RGB.exe, we’re presented with three sliders, presumably corresponding to the RBG colors, and once you’ve set their values you can check them. This indicates that we’ll need to figure out the correct three values to access the key.

Wrong values

 

A quick look at the PE file with Exeinfo shows that it’s a .NET program, which can be unpacked with de4dot.

 

Checking binary type

 

Running de4dot against the executable creates a new file, “RGB-cleaned.exe” that we can then decompile with dnSpy to look at the underlying source code.

Deobfuscating binary

 

When looking at the source code, we come to the challenge that we’ll need to solve.

Algorithm

Simply put, three conditions need to be met to get the MessageBox we want to display. At this point, I started poking around to see if I can just modify the code so it always prints the answer, but when you start diving into the functions being called, you can see there is a bit more going on and requires the actual numbers.

XORing numbers against array of numbers

So I decided to tackle the math aspect instead.

The three conditions that need to be met are that one equation result must equal another equation result and one of the specific values needs to be over 60. I opt to brute force it by iterating through every possible combination of numbers, knowing that each slider will be in a range of 1-255, with one being in the range of 60-255. This gives us roughly 12.5 million possibilities, 255255(255-60), which shouldn’t take long at all.

After a few minutes of thinking through the logic, I use the below script to find the value.

import sys


def func1(value, value2, value3, num, num2):
    result = value + num - value2 + value * value * value2 - value3
    return result


def func2(value, value2, value3, num, num2):
    result = value2 * (value3 * 34 + (num2 - value)) + 3744
    return result


def check(value, value2, value3, num, num2):
    a = func1(value, value2, value3, num, num2)
    b = func2(value, value2, value3, num, num2)
    if a == b:
        print "\nwinner winner, chicken dinner!\n\n", value, value2, value3, " [" + str(
            a), str(b) + "]"
        sys.exit(1)
    return


value = 60
value2 = 0
value3 = 0
num = value2 * value3
num2 = value * 3

while value < 256:
    value2 = 0
    value3 = 0
    num = value2 * value3
    num2 = value * 3
    check(value, value2, value3, num, num2)
    while value2 < 256:
        value3 = 0
        num = value2 * value3
        num2 = value * 3
        check(value, value2, value3, num, num2)
        while value3 < 256:
            num = value2 * value3
            num2 = value * 3
            check(value, value2, value3, num, num2)
            value3 += 1
        value2 += 1
    value += 1

 

Within a few seconds, we have the answer and can validate it within RGB.exe to get the key.

PAN{l4byr1n7h_s4yz_x0r1s_4m4z1ng}