Let me try and Fixit!

A few years ago, I built a Burp plugin called FixerUpper in an attempt to make life testing FIX systems easier. However, not long after releasing that tool I realized that it just didn’t quite cut it, and I needed to build something... better.

I started from the ground up and have spent the past “while” building and testing my own custom FIX client for use during testing. The aim was to allow more flexible interaction with FIX systems and offer a wider range of the features required to actually perform security testing, rather than just talk FIX.

I called this new tool “Fixit”, and you can download it here: https://github.com/WithSecureLabs/fixit.

Read on to learn about:

• Why Fixit was created
• Some key requirements for successful FIX system testing
• How to use Fixit to craft FIX messages and perform testing

Back in 2020, I decided to do some research into the FIX protocol and FIX-based systems. A logical direction was to create a Burp plugin, integrating into something many people are familiar with. This led to the FixerUpper Burp plugin being created, and it served its purpose well… for a bit…

A few years have passed now, and I’ve done quite a bit more testing of FIX-based technologies. As a result it’s become apparent that FixerUpper just isn’t flexible or robust enough. It has too many limitations which I hadn’t fully considered at the time. One of which is that FIX is asynchronous, so being unable to handle and process heartbeat messages whilst simultaneously intercepting and modifying other messages is a big problem. It also requires you to set up mitm_relay.py, which is an extra step - and no one wants extra steps!

During the development of “Fixit”, it’s become clear that building a robust FIX client with the functionality and features to help with security testing is significantly harder than building a tool just to send pre-determined FIX messages. Especially as each assessment led me to encounter different FIX specifications and edge cases. But now after working on it for a while, and testing it on a range of assessments, it is time to release it to the world. This way, it can be put through its paces even more, and hopefully improved by a wider community.

Fixit is ultimately just a custom command line FIX client application. It just happens that this application is made with crafting, manipulating, and fuzzing of FIX messages directly in mind - rather than trying to correctly ingest market data or accurately push orders / process execution reports, etc with the messages themselves abstracted away from the user.

It is based on the QuickFix python library, with a significant amount of "wrapper" around it to make it more appropriate for security testing. It provides more practical features and addresses some limitations of the library that – as testers – we really don’t want.

But anyway! Instead of talking about it and listing off features, I'll walk through using it and explain things as we go!

Requirements for testing and initial access

When approaching a FIX system, the three main pieces of information we really want are:

1. A valid FIX initiator configuration file for the system: This lets us know exactly what client properties are needed to successfully establish a connection to the server.
   
   
2. The FIX specification used: This lets me see what messages the system is designed to process, and exactly how they should be structured.
   
   
3. Examples of valid FIX messages sent to/from that system: This lets me see real valid messages, so I know for a fact if the messages I generate should be valid or not. I’d argue the LOGON (A) message is the most important, as if you can’t get that right you’ll be stuck for a while.

Establishing initial access using the Fixit initiator

To establish a connection to a FIX serve, a valid initiator configuration file is required. This file is used by FIX clients to specify the required properties to establish a valid and stable connection to a FIX system. Without it, you won’t be able to communicate with the target FIX server.

For example, consider the following server (FIX “acceptor”) configuration for FIXimulator:

Among other things, this configuration specifies that the version of FIX used is FIX.4.2 and that the heartbeat interval for connections is 30 seconds. Additionally, it outlines a session config that will accept connections on port 9878 with a Target and Sender CompID combination of SOMECLIENT->FIXIMULATORTRADE.

If incorrect values are specified within the client (FIX “initiator”) configuration file, the connection will fail, and it often won’t be obvious why. For example, we could attempt to connect using the following initiator config, where the Heartbeat Interval, FIX version, and SenderComID values are wrong:

Using the Fixit initiator.py script to try and establish a connection using this configuration, it can be observed that after three attempts, the logon fails. However, the protocol does not provide much (or often any) information as to why:

And with this, we’re left wondering: Is the server running correctly? Does it require a username? Was the FIX version correct? Was the Sender / TargetCompID correct? Maybe the heartbeat interval was wrong? Or does it expect the connection to start with a different sequence number? Too many unknowns to reasonably address!

This is why it’s important to have access to a valid initiator configuration file, and ideally an example valid LOGON (A) message. This lets us troubleshoot and compare what’s being observed with what is known to be expected. Updating the initiator configuration to be contain valid properties for the target system, re-running the initiator.py script should successfully exchange LOGON (A) messages with the server and establish a FIX session:

After the connection is established, access can quickly be tested using the MESSAGE SEND TEST command. This will send a simple FIX Test Request (1) message to the server, and if successful, a Heartbeat (0) message should be received in response.

This (and all subsequent) exchange of messages can be investigated using the HISTORY command:

Messages in the application’s history can be inspected using the HISTORY VIEW command. Below this is used to confirm that the Heartbeat (0) returned contained the appropriate TestReqID (112), corresponding to the test message sent:

OK! So that’s how to test initial access to a FIX server. Next it’s time to craft some FIX messages!

Crafting valid FIX messages from scratch

In the event there is no knowledge of the FIX message specification for the target system, and there is no access to sample messages, Fixit can generate a select few basic message types automatically. This is achieved using the MESSAGE NEW command:

This generated message may or may not be successfully processed when sent to the server, but it’s a starting point if we have no other information. That said, it’s far easier when we know what type of messages we should send!

Crafting valid FIX messages from a specification

So, what about if a FIX specification is available for the target system? In this scenario, we can easily craft a message manually. Consider the following specification for a NewOrderSingle (D) message:

This can be references to manually craft a known-to-be-valid NewOrderSingle (D) message for the target system - as well as any other messages defined within the specification. This can be as simple as writing the field=value pairs in a text file:

Note: If you want more information on the anatomy of FIX messages, the protocol architecture, the authentication sequence, etc. You can find this in my previous “FixerUpper” blog post here.

This process is a bit cumbersome, but once completed the messages can be loaded directly into Fixit using the MESSAGE LOAD FILE command:

Message strings can also be loaded in ad hoc via the MESSAGE LOAD STRING command:

As mentioned, composing raw messages by hand is generally painful, but if no examples are available, it may be the only option.

Crafting valid FIX messages using examples

A better scenario is if we can get some example messages! This could be a log dump from the server containing FIX messages (often in a similar format to the text file we just created), or even a network capture. Assuming we’ve ASCII or binary formatted FIX message data, they can be loaded into Fixit using MESSAGE LOAD FILE, extracting any message it can find in the file:

All messages loaded into Fixit are stored in the message store and can be inspected using the MESSAGE LIST:

Manipulating messages using Fixit

Now that we have a bunch of messages, let’s see how Fixit can help do some actual testing by modifying message fields.

Modifying basic message fields:

To begin exploring how to manipulate and send created or imported messages, let’s filter them to find ones we want. For example, if we just want to see different versions of the New Order (D) message, they can be filtered using expressions:

With the ID’s listed, we can select a message to work with using the MESSAGE USE command and inspect it current composition using MESSAGE VIEW. This displays the message in a more human readable format that makes it easier to understand / edit:

To begin with, we’ll just try and send a valid message placing an order on the target system. By typing MESSAGE EDIT on its own, Fixit will open the message in a default text editor and allow you to freely edit and save it:

After making changes, we can simply save and exit to commit the changes. In this case, the message was updated to form a “Market Order” for 500 units of the asset identified by the ticker symbol THQI:

After saving, the changes can be inspected by running MESSAGE VIEW again, where we can check it updated successfully:

With the desired message crafted, it can be sent to the server using MESSAGE SEND. Once sent, and a response has been received (or not), the most recent message activity can again be reviewed using the history HISTORY command:

This shows us that the New Order (D) message was sent out y the application, followed by an Execution Report (8) being received. Looking at the FIXmulator GUI directly, it can be confirmed that a new Buy order has indeed been created:

Once the server executes the order, a response notification will be displayed in the application interface. The command HISTORY VIEW -1 raw can be used to inspect the last message in the session history in its raw format. The “order status” value  - represented by field 39 – informs the client that that the order was filled (2):

A faster way to edit messages is to use MESSAGE EDIT command non-interactively. This can be achieved by specifying the fields we want to update (NUM=VAL), remove (-NUM), or insert (+POS:NUM=VAL) as command arguments:

The changes above change the original “Buy Order” into a “Limit Sell Order”. Once sent, this message creates an instruction to sell 250 units the asset when the price reaches 70.00 using the following fields / values:

Reusing message fields:

Sometimes after creating an order, it may be necessary to cancel it or request a status update. These types of messages require information regarding the original order; however it would be a pain to need to manually copy and paste them. With this in mind, Fixit allows for the creation of messages using a reference to a previous order within the session history.

For example, using the MESSAGE NEW ORD-CANCEL command can be used to create a new OrderCancelRequest (F) for a specified order. This is done by referencing the Fixit message history ID of the original order within the command:

When the cancel request has been executed by the FIX system, an Execution Report (5) will be received. This message will contain an “Order Status” (39) field informing the client if the cancel was successful or not. This and other relevant fields can be extracted from the response using the FILTER operator of HISTORY VIEW command:

This response shows that the original sell order for 250 units was successful cancelled - as broken down below:

As some commands (such as message edit) can get a quite complex, the Fixit CLI offers help and example usage for each command. For example, the help for MESSAGE USE can be seen below:

Investigating a basic test scenario:

So far, we’ve seen how to create legitimate FIX traffic, but when testing, a lot of what will be performed is actually trying to create non-standard / invalid / illegitimate FIX traffic to see how it’s handled by the server. For example, a basic test scenario would be to see how the system processes negative prices. This can easily be achieved by creating a new message and setting a negative price using the commands shown so far:

This is also a good time to introduce “command preloading” using the --preload command line argument. This feature is extremely handy and allows you predefine as many commands as you want for the application to run automatically as soon as it starts. For example, this feature can be used to run the initiator and automatically, load and select a message, alter its field, send it, parse the response, and finally gracefully exit - all in one command, without the need for interaction:

However, other test cases might require the sending of binary data to see how the server handles that. For this, the tool offers the MESSAGE FUZZ command. This command accepts a wordlist of payloads, or if one is not provided, generates a basic fuzzing payload using byte values from 00 to FF. For example, if we want to test how the server processes unexpected binary data within the Symbol (55) field using MESSAGE FUZZ 55:

Once fuzzing is complete the results will be stored in a CSV file under /output/ for review:

The speed of the fuzzing feature is limited by the application’s fuzz_delay and resp_delay options. These control how long the application waits for message responses as well as the delay between message fuzzing loops. By default, these are set to 0.5 and 0.1 seconds respectively. However, depending on how long the wordlist is, it might need to run faster. This can be achieved using the --fuzz_delay and --resp_delay command line arguments for the initiator.py script.

A complication that may arise with fuzzing is that a value in the payload may break the message syntax. This is where the application provides the ability to change the delimiter for the FIX message fields (aka the Start of Header or SOH character). This is important if your payload contains the pipe (|) character for example, which is the default SOH value for viewing messages in Fixit.

Below, another fuzzing sequence is performed using an SQLI dictionary. However, Fixit is flagged to use tilde (~) character as the SOH instead of pipe (|) as the wordlist payloads contain the pipe character. If this was not performed, Fixit would fail to correctly parse the messages as it relies on the ASCII SOH character to know where fields begin and end:

It should be noted that the actual value of the SOH in FIX messages once released is 0x01. However, for readability and usability it when presenting message it uses “|” in the CLI, or any value set using --fix_delim.

Modifying critical message fields:

A more difficult test scenario is the modification of critical message fields. The fix protocol requires that messages contain several header and trailer fields, containing some of the protocol specific information in them.

For example, the message headers contain fields like BeginString (8) which specifies the FIX version of the message and SenderCompID (49) which identifies the sending organisation. These values are strictly tied to the socket connection, and even the QuickFIX APIs won’t allow for the modification of these values once a session is established. 

This behaviour can be highlighted by attempting to edit the SenderCompID (49). After an edit, inspecting the message confirms that the field is set to TEST and - so far - everything seems fine:

However, when attempting to send the message this will be overridden as the, the SenderCompID forms part of the FIX session, and the FIX session is tied to the current socket. This is defined by the combination of BeginString:SenderCompID->TargetCompID. In this case the session structure is:

FIX.4.2:SOMECLIENT->FIXIMULATORTRADE

So, what happens if we do try to send the message?

As warned, SenderCompID (49) is reverted! This is not done by Fixit, but instead a side-effect of controls enforced by the underlying QuickFIX APIs. As this modification should never happen, when a message is released, it updates these fields with those found in the session state, effectively ignoring any user-made changes to them.

To get around this, Fixit includes a feature called the “interceptor”. This component listens for FIX messages leaving the application and intercepts them. Then, if required, it replaces the message data with the original user-defined raw message data, finally forwards that onto the server, bypassing the API modifications.

Thankfully it’s not a separate tool you need to run, and it’s built into Fixit directly. The only thing requirement to enable this feature is to set the following properties within the initiator configuration file (I always run Fixit with the interceptor configured):

Now when Fixit starts, a new notification will be listed during startup, showing that an interceptor has been created:

To trigger use of the interceptor, the RAW modifier for the MESSAGE SEND command is used. This will ensure that the raw payload reaches the server regardless of what values it holds:

However, although the modifier allows for an invalid CompID to easily be sent, there are still multiple fields that the FIX protocol relies on to process messages correctly. Even when using the RAW modifier these fields are still updates for the user in order to ensure the message can be processed.

This automated “patching” is not desirable if we’re trying to test how these fields are processed though. As such, additional modifiers can be included to prevent the automated update of these fields:

• -US: Prevent the automated updating of the SeqNum (34) field
• -UL: Prevent the automated updating of the BodyLength (9) field
• -UT: Prevent the automated updating of the SendingTime (52) field
• -UC: Prevent the automated updating of the Checksum (10) field

By combining these modifiers it allows the tester to freely manipulate these fields, without them being patched by the tool:

Inspecting the Fixit message logs:

Something else worth discussing quickly is the application’s message logs!

If the app crashes or you close the application by accident and you want to see the message history from your last session, the message logs can be found in the /log/ directory. This includes native QuickFix logs as well as Fixit-specific message logs:

The Fixit message log files end in “*fixit.message.log” and include all messages sent and received by the Fixit application, as well as notes on any original and intercepted messages for where the interceptor was triggered:

So that’s the Fixit tool! There are a bunch of other features, commands, and specific scenarios, but that’s enough to get people started. As a disclaimer, I wouldn’t consider it “finished” or exactly where I’d want it to be. There is plenty of work to do!

For starters, the Fuzzing feature could probably be rewritten from scratch in a much better and flexible way using established fuzzing libraries / frameworks. The main challenge is integrating these things with the active FIX socket connection, and if sending raw data, ensuring minimal FIX-protocol specifics are implemented to ensure the messages actually get processed.

There are also many edge cases I’ve found on various client engagements where additional fields are required by their custom specification during the logon exchange but they that aren’t FIX-native or understood by the current build. In these situations, I’d find myself needing introduce ad hoc code changes to include them. Ideally there would be a way to pass a list of non-standard fields that the tool so it could process them and know to include them where needed.

Please feel free to try the tool out, make improvements, or log bugs / issues for review.

A specific appreciation for the research and effort put in to pre-existing tools such as Fixer (SECFORCE) and Fizzer (AonCyberLabs) which inspired some of my work.

Also, a big shout out to Zoltan Feledy for creating FIXimulator! As this was the only access to any kind of “FIX server” I could get when I was between FIX-specific projects, allowing me to keep working on the tool!

Withsecure.com – A bit of a Fixer Upper - Testing FIX-backed applications

quickfixengine.org – QuickFix Library

fiximulator.org – FIXimulator & Banzai

onixs.biz – FIX Documentation

github.com – fixer (SECFORCE)

github.com – Fizzer (AonCyberLabs)

giac.org – Exploiting Financial Information Exchange FIX Protocol

fixtrading.org – Fix Security Whitepaper v1.9