Part 2 of Semmle QL’s vulnerability hunting

Semmle QL and the Microsoft Security Response Center’s ( MSRC ) use of it to look into different types of vulnerabilities that have been reported to us were discussed in the first part of this series. This article covers a security audit of an Azure firmware component as an illustration of how we’ve been proactively using it.

This was part of a larger defense-in-depth security review of Azure services that examined attack vectors from the perspective of an alleged foe who is currently residing in the service backend’s operating environment ( marked with * on the diagram below ) after breaching at least one security boundary.

A Linux-based embedded device that connects to both a service and management backends and transmits operational data between the two was one of the review’s objectives. This device’s management protocol, which is used on both interfaces, serves as its primary attack surface.

This management protocol is message-based, according to an initial manual review of its firmware, and there are more than 400 different message types, each with a handler function. Using Semmle to expand our code review capabilities was a simple decision because manually auditing every single function would have been time-consuming and error-prone. Using the static analysis methods covered in this post, we discovered a total of 33 vulnerable message handler functions.

Identifying the attack’s surface

To model data that would be obtained from an attacker, we first wrote some QL. Every message request type is identified with a category number and command number according to the management protocol’s request-response system. Arrays of structures like these are used to define this in the source code:

CMD_CATEGORY_BASE, g_Command Handlers_Base, cmd_cated_category_app0, mg_commendhandler’s appoint, null, NULL,

g_Command Handlers_Base] ] = CMD_GET_COMPONENT_VER, sizeof ( ComponentVerReq ), GetComparticipVer,…,…- GET-GLOBAL_CONFIG,-1, GetGlobalConfig…

In the example above, a message with category type CMD_CATEGORY_BASE and command type CMD_GET_COMPONENT_VER would be routed to the GetComponentVer function. The command handler table also has information on the expected size of the request message, which is validated in the message dispatch routines prior to calling the handler function.

The following QL was used to define the message handler table:

class CommandHandlerTable extends Variable { CommandHandlerTable() { exists(Variable v | v.hasName("g_MessageCategoryTable") andthis.getAnAccess() = v.getInitializer().getExpr().getAChild().getChild(1) ) } }

This takes a variable named g_MessageCategoryTable, finds its initializing expression, and matches all children of this expression – each child expression corresponds to a row of the message category table. For each row, it takes the second column (this is getChild(1) because the parameter of the getChild predicate is zero-indexed), each of which are references to a command handler table, and matches on the variable referenced. In the example above, these would be g_CommandHandlers_Base and g_CommandHandlers_App0.

We used a similar methodology to define the message handler functions:

Class MessageHandlerFunction extends Function Expr tableEntry,

( Command Handler Table table| tableEntry = table ) is a function that is used by the Message HehandlerFunction. ( ) getInitializer ( )getExpr This = tableEntry and getAChild ( ). ( 2 )getChild ( FunctionAccess ). ( ) getTarget

tableEntry = int getExpectedRequestLength ( ) result. ( 1 )GetChild getValue( ). ( ToInt ) )

…}`

This QL class uses a member variable tableEntry to hold the set of all rows in all command handler tables. This is so it can be referenced in both the characteristic predicate (MessageHandlerFunction() { … }) and getExpectedRequestLength(), without repeating the definition.

All of this corresponds to the aforementioned code structure:

The signature for each message handler function is the same:

Unsigned charUINT8 typedef

ExampleMessageHandler ( UINT8 pRequest, RequestLength, UITE8PResponse )

and adheres to a general pattern in which the message layout-representing struct type receives the request data and can be accessed through its fields:

Examples of Messages are listed in ExampleMessageHandler ( UINT8 ) pRequest, RequestLength, and UINT8PResponse ) as follows:

…

someFunction(pMsgReq->aaa.bbb)

…}`

In this analysis, we were only interested in the request data. We defined two additional predicates in the MessageHandlerFunction QL class to model the request data and its length:

Class MessageHandlerFunction extends Function Expr tableEntry,

…

This is the result of the parameter getRequestDataPointer( ). ObtainParameter ( 0 ).

This is the result of the parameter getRequestLength( ). Obtain Parameter ( 1 )

A message handler function can be used in the same way as any other QL class after its definition has been abstracted away. For instance, according to the cyclomatic complexity of each message handler function, this query lists them all:

from MessageHandlerFunction mhf select mhf, mhf.getADeclarationEntry().getCyclomaticComplexity() as cc order by cc desc

examining the flow of data

The next step was to determine where untrusted data might be used in an unsafe way now that we had established a set of entry points. We had to monitor how such data moved through the codebase in order to accomplish this. The majority of the challenging language-specific detail involved in this is abstracted away by QL’s robust global data flow library.

The DataFlow library is brought into the scope of the query with:

import semmle.code.cpp.dataflow.DataFlow

It is used by subclassing DataFlow::Configuration and overriding its predicates to define the data flow as it applies to DataFlow::Node, a QL class representing any program artefact that data can flow through:

The majority of data flow queries will have the following appearance:

DataFlow is extended by the class RequestDataFlowConfiguration. Configuration :RequestDataFconfiguration ( ) this equals” CustomerDataFloConformation.”

Source ( DataFlow:: Node source )… is the overridepredicate.

DataFlow’s overridepredicate isSink ( Node sink )…

AdditionalFlowStep ( DataFlow:: Node node1, DataFlow, :… is the overridepredicate.

Barrier ( DataFlow:: Node node )… is the overridepredicate.

}

Node source, node sinkwhereany ( RequestDataFlowConfiguration c ) from DataFlow ” Data flow from$ @ to$$ @” is selected by hasFlow ( source, sink ).

The QL data flow library performs an interprocedural analysis, which includes data flowing through function call arguments in addition to looking at data flows specific to a function. This was a crucial aspect of our security review because, despite the fact that the weaker code patterns discussed below are easily demonstrated using simple example functions, the majority of the results came from the actual source code for our target.

identifying weaknesses in memory safety

We started by looking for code patterns relating to memory safety because this firmware component was written entirely in C.

Array indexing without running a bounds check is one frequent source of these bugs. Since the attacker’s level of control over the index value is what we are really interested in, searching for this pattern alone would yield a significant number of results that are most likely not security vulnerabilities. Therefore, in this instance, we are looking for data flows with an array indexing expression serving as the sink, a message handler function’s request data as its source, and any data flow node being protected by an appropriate bounds check.

For instance, we’re looking for data flows that match the following code:

Int ExampleMessageHandler ( UINT8 pRequest ( 1 ), Source ), int RequestLength, and tint ExamplesMrequestpMsgReq ( 3 ), i .e., index1 ( 6 ) = msmreq ( 4 )- &gt ( 5 ), etc.

Index 1 ( 7: sink ) in pTable1]. field1 = pMsgReq-&gt,value1

However, we also want to keep these types of code’s data flows out:

Int ExampleMessageHandler ( UINT8 pRequest ( 1 ), Source ), int RequestLength, and tint Index2(6 )- &gt, index2(5 ), exemplified by the expressions” ExampleMessageRquestpMsgReq( 3 )” and” ExpatumMessageRequeque” ( explained in the following manners:

If ( index2 &gt, = 0 amp, index2, andlt ( PTABLE_SIZE), then index2 is true. field 1 = pMsgReq-&gt,value2

The source is defined using the MessageHandlerFunction class discussed earlier, and we can use the getArrayOffset predicate of an ArrayExpr to define a suitable sink:

Any ( MessageHandlerFunction mhf ) overridepredicate isSource ( DataFlow:: Node source ). source = getRequestDataPointer ( ). ( AsParameter ) )

There is an overridepredicate called isSink ( DataFlow: Node sink ) and it is ( ArrayExpr ae | Ae ). sink = getArrayOffset ( ). ( asExpr ) )

By default, the DataFlow library only includes flows that preserve the value at each node, such as function call parameters, assignment expressions, and the like. But we also need data to flow from the request data pointer to the fields of the structure it was cast to. We’ll do that like this:

overridepredicate isAdditionalFlowStep(DataFlow::Node node1, DataFlow::Node node2) { // any terminal field access on request packet // e.g. in expression a->b.c the data flows from a to c exists(Expr e, FieldAccess fa | node1.asExpr() = e and node2.asExpr() = fa | fa.getQualifier*() = e andnot (fa.getParent() instanceof FieldAccess) ) }

Any node with a variable or field used earlier in the control flow graph ( for the time being, we assume that any such bounds check is carried out correctly ) is barred in order to prevent flows that have been bound:

overridepredicate isBarrier(DataFlow::Node node) { exists(ConditionalStmt condstmt | // dataflow node variable is used in expression of conditional statement // this includes fields (because FieldAccess extends VariableAccess) node.asExpr().(VariableAccess).getTarget().getAnAccess() = condstmt.getControllingExpr().getAChild*() // and that statement precedes the dataflow node in the control flow graph and condstmt.getASuccessor+() = node.asExpr() // and the dataflow node itself not part of the conditional statement expression andnot (node.asExpr() = cs.getControllingExpr().getAChild*()) ) }

The data flow through each node in the two examples above would be as follows:

This query discovered 18 vulnerabilities across 15 message handler functions in our firmware codebase, including attacker-controlled out-of-bounds reads and writes.

We applied a similar analysis to find where arguments of function calls were taken from the message request data without first being validated. Firstly, we defined a QL class to define the function calls and arguments of interest, including the size argument of calls to memcpy and a similar function _fmemcpy, and the length argument of CalculateChecksum. CalculateChecksum is a function specific to this codebase that would return the CRC32 of a buffer, and could be potentially be used as an information disclosure primitive where the message handler function copied this value into its response buffer.

FunctionCall extends int argToCheck in the class ArgumentMustBeCheckedFunctionCall.

( this )ArgumentMustBeCheckedFunctionCall ( ) getTarget argToCheck = 2 ) and hasName ( “memcpy” ), or ( this ). ( ) getTarget argToCheck = 2 ) and hasName ( “_fmemcpy” ), or ( this ). ( ) getTarget argToCheck = 1 and hasName ( “CalculateChecksum” )

This is the output of Expr getArgumentToCheck( ). argToCheck ( getArgument )

Next, we modified the sink of the previous query to match on ArgumentMustBeCheckedFunctionCall instead of an array index:

overridepredicate isSink(DataFlow::Node sink) { // sink node is an argument to a function call that must be checked first exists (ArgumentMustBeCheckedFunctionCall fc | fc.getArgumentToCheck() = sink.asExpr()) }

In 13 message handlers, this query found an additional 17 vulnerabilities, the majority of which were attacker-controlled out-of-bounds reads ( for which we later confirmed it was disclosed in a response message ).

Tracking taint

In the above queries, we overrode the DataFlow library’s isAdditionalFlowStep predicate to ensure that where data flowed to a pointer to a structure, the fields of that structure would be added as nodes in the data flow graph. We did this because by default, the data flow analysis only includes paths where the value of the data remains unmodified, but we wanted to keep track of a particular set of expressions that it may have affected too. That is, we defined a particular set of expressions that were tainted by untrusted data.

QL contains a built-in library to apply a more general approach to taint tracking. Developed on top of the DataFlow library, it overrides isAdditionalFlowStep with a much richer set of rules for value-modifying expressions. This is the TaintTracking library, and it is imported in a similar manner to DataFlow:

import semmle.code.cpp.dataflow.TaintTracking

It is used in almost the same way as the data flow library, except that the QL class to extend is TaintTracking::Configuration, with these configuration predicates:

We re-ran the earlier queries with isAdditionalFlowStep removed (as we no longer need to define it) and isBarrier renamed to isSanitizer. As expected, it returned all the results mentioned above, but also uncovered some additional integer underflow flaws in array indexing. For example:

Int ExampleMessageHandler ( UINT8 pRequest ( 1 ), Source ), int RequestLength, and tint ExamplesMrequestpMsgReq ( 3 ), i .e., index1 ( 6 ) = msmreq ( 4 )- &gt ( 5 ), etc.

( index1 ( 7 )- 2 ) ( 8: sink )pTable1 ] field1 = pMsgReq-&gt,value1

For our internal reporting of each vulnerability type, we were interested in classifying these separately from the earlier query results. This involved a simple modification to the sink, using the SubExpr QL class:

overridepredicate isSink(DataFlow::Node sink) { // this sink is the left operand of a subtraction expression, // which is part of an array offset expression, e.g. x in a[x - 1] exists(ArrayExpr ae, SubExpr s | sink.asExpr() instanceof FieldAccess and ae.getArrayOffset().getAChild*() = s and s.getLeftOperand().getAChild*() = sink.asExpr()) }

As a result, the two message handler functions now have 3 additional vulnerabilities.

identifying vulnerabilities in path traversals

We used QL to try and find message handler functions that used an attacker-controlled filename in a file open function in order to find potential path traversal vulnerabilities.

This time, we took a slightly different approach to taint tracking, defining some additional steps that would pass through different string-processing C library functions:

Exists ( FunctionCall fc, Function |expSrc = cc ) as the predicate of isTaintedString ( Expr expSlc vs. expDest ) fc = getArgument ( 1 ) and expDest. fc = getArgument ( 0 ) andf. ( f. hasName ( “memcpy” ) orf and get Target( ). hasName ( fmemcpy )orf hasName ( memmove )orf hasName ( strcpy )orf hasName ( strncpy )orf hasName ( strcat )orf HasName ( “strncat” ) ) orexists ( ExpSrc = fc, Function C, int n ). fc = getArgument ( n ) and expDest. fc = getArgument ( 0 ) andf. GetTarget ( ) and ( (f.hasName ( “sprintf” ) and n &gt, = 1 ) or ( 4 ) and ( 2 ), respectively.

…

The overridepredicate isAdditionalTaintStep ( DataFlow: Node node1 ), DataFlow; Nodes nodes ( 2 ), and isSaintedString ( node1 ). asExpr ( ), node2. ( asExpr ) )

and described the sink as a file open function’s path argument:

This is a function that the class FileOpenFunction extends. hasName ( fopen )orthis HasName ( open ).

result = 0 // filename parameter index int getPathParameter ( )

…

There is an overridepredicate for isSink ( DataFlow: Node sink ) ( FunctionCall fc, FileOpenFUNction fof |ff ). Fof and fc are equal to getTarget ( ). GetArgument ( fof ). Sink = getPathParameter ( ) ). ( asExpr ) )

Before tackling the next issue of excluding flows where the data was validated, as with the earlier queries, we anticipated at least some results because we had some prior knowledge of how our target device operated based on an initial review. However, there was absolutely no response to the query.

We resorted to querying the function call graph to look for any path between message handler functions and a call to an open file function, excluding calls where the path argument was constant:

This recursive predicate defines the function mayCallFunction, which is a function caller and functioncall fc. Caller or mayCallFunction (fc ) = getEnclosing( ). ( ), fc ) getTarget

MessageHandlerFunction ( mhf, fc ), FunctionCall( ), FileOpenFUNction fofwhere may, and maycall( ) are examples of these functions. Fofandnot fc = getTarget ( ). GetArgument ( fof ). ( )getPathParameter “$ @ may have a path to$ @,” isConstant ( ) selectmhf. Fc, fc ( ), toString ( ). ( )toString

Five results from this query—enough to be manually examined—led to the discovery of two path traversal vulnerabilities—one for writing to files and one for reading from files—both of which had an attacker-supplied path. Due to the need to send two different message types—one to set the filename and the other to read or write data to files with that name—it turned out that taint tracking did not flag these. Fortunately, QL was adaptable enough to allow for a different exploration path.

Conclusions

Microsoft takes a comprehensive defense strategy to protect the cloud and the data of our customers. Conducting thorough security reviews of Azure internal attack surfaces is crucial to this. We used Semmle QL’s cutting-edge static analysis techniques to identify flaws in a message-based management protocol in this source code review of an embedded device. 33 message handlers in total, belonging to various bug classes, were found as a result. We were able to use QL to automate repetitive tasks from a manual code review while still using an explorative methodology.

MSRC Vulnerabilities &amp, mitigation team Steven Hunter and Christopher Ertl