Pharo Libclang FFI, part 5, client data and recursive visitor/callbacks

Now we make use of the client data to track the indent level.  The recursive call to clang_visitChildren() seems a bit of an anti-pattern to use with a visitor – presumably a new visitor is created each call.   However that’s how it was done in a few tutorials I found and it does provide local storage for each nextLevel variable for the purpose of this demonstration.

Doing it in C

In simple.c, change acceptCursorCallback as follows…

#include clang-c/Index.h
#include stdio.h

void show_cursor_kind(CXCursor cursor)
    enum CXCursorKind cursorKind  = clang_getCursorKind(cursor);
    CXString kindName  = clang_getCursorKindSpelling(cursorKind);
    CXString entityName = clang_getCursorSpelling(cursor);
    printf("%03d  %s(%s)\n",

// accept cursor callback
enum CXChildVisitResult
acceptCursorCallback(CXCursor cursor,
                     CXCursor parent,
                     CXClientData clientData)
        int curLevel  = *( int *) clientData;
        printf("%.*s", curLevel, "--------------------------------");
        int nextLevel = curLevel + 1;
        clang_visitChildren( cursor, acceptCursorCallback, &nextLevel );
        return CXChildVisit_Continue;

int main( int argc, const char *const * argv )
{   // Process arguments
    if( argc < 2 )
    {   printf("Usage: %s inputfile {clang-options}\n", argv[0] );
        return -1;
    const char * mainSourceFilename = argv[1];
    const char *const* options = argv + 2;
    int optionCount = argc - 2;

    // Create the Index and Transalation Units
    CXIndex index = clang_createIndex( 0, 0 )
    CXTranslationUnit TU =
            clang_createTranslationUnit(index, mainSourceFilename);
    if( !TU )
    {   printf("Failed to get Translation Unit\n");
        return -1;

    // The root cursor of a TU is the translation unit itself
    CXCursor rootCursor  = clang_getTranslationUnitCursor( TU );

    // invoke visitor
    int treeLevel = 1;
    clang_visitChildren(rootCursor, acceptCursorCallback, &treeLevel);

    clang_disposeTranslationUnit( TU )
    clang_disposeIndex( index );
    return 0;

Compile and run and we get…
$ clang-3.5 -I/usr/lib/llvm-3.5/include -lclang -o simple simpl.c
$ ./simple foo.ast
Processing foo.ast
300 TranslationUnit (/home/ben/Apps/moose_suite_6_0/libclang-play/foo.c)
-008 FunctionDecl (addtwo)
–010 ParmDecl (number)
–202 CompoundStmt ()
—214 ReturnStmt ()
—-114 BinaryOperator ()
—–100 UnexposedExpr (number)
——101 DeclRefExpr (number)
—–106 IntegerLiteral ()
-008 FunctionDecl (main)
–202 CompoundStmt ()
—231 DeclStmt ()
—-009 VarDecl (result)
—–103 CallExpr (addtwo)
——100 UnexposedExpr (addtwo)
——-101 DeclRefExpr (addtwo)
——106 IntegerLiteral ()

Okay, lets try this in Pharo….

First attempt in Pharo

One potential complication I foresaw was that for different applications, CXClientData will need to hold different data. In C, its definition as a void pointer is compatible with any pointer type, and cast to the required type in the callback function.  The question is how to achieve the same casting facility with Pharo FFI.

Since the Pharo object passed via callout to clang_visitChildren()  is passed straight to the Pharo callback block, ideally any normal Pharo object could be put inside some FFI wrapper and unwrapped inside the callback.

After spending a while looking a various alternatives I managed to work out a tentative path forward. Its a bit low level, but at least demonstrates what I’m trying to achieve, and perhaps will help lead to a better solution later. In particular, the #basicXXX methods are marked ‘private’ so there should be a better way. But for now just an experiment to make progress….

intPtrType := FFIExternalType resolveType: 'FFIUInt64'.
pointer := ExternalAddress allocate: intPtrType externalTypeSize.
pointerCopy := pointer asInteger.
type handle: pointer at: 1 put: 42.
TestCase assert: pointer asInteger = pointerCopy.
TestCase assert: 42 = (intPtrType handle: addr at: 1).

So the first assert shows that the pointer wasn’t clobbered by the #handle:at:put: and the second assert shows the value retrieved is the same as that stored, so it correctly used the dereferenced memory space. Lets implement that in a test case.

LiblangTest >> visitChildrenClientData: callOutClientData
  | rootCursor acceptCallbackFn callbackClientData |
  rootCursor := self getRootCursor.
  acceptCallbackFn :=
        signature:  #( CXChildVisitResult (
			        CXCursor cursor,
				CXCursor parent,
				CXClientData client_data))
        block: [ :cursor :parent :clientData |
			callbackClientData := clientData.
		 	CXChildVisit_Break value ].

    Libclang clang_visitChildren__parentCursor: rootCursor
                             visitorCallback: acceptCallbackFn
                                  clientData: callOutClientData.
  ^ callbackClientData
LiblangTest >> testVisitChildrenClientDataMultipleTypes
  | clientDataType calloutPointer callbackPointer value | 	

  "Client data as an integer"
  clientDataType := FFIExternalType resolveType: 'FFIUInt64'.
  calloutPointer := ExternalAddress allocate: clientDataType externalTypeSize.
  clientDataType handle: calloutPointer at: 1 put: 42.

  callbackPointer := self visitChildrenClientData: calloutPointer.
  self assert: calloutPointer equals: callbackPointer.
  value := clientDataType handle: callbackPointer at: 1.
  self assert: value equals: 42.

  "Client data as a structure"
  clientDataType := FFIExternalType resolveType: 'LiblangTestStruct'.
  calloutPointer := ExternalAddress allocate: clientDataType externalTypeSize.
  value := LiblangTestStruct fromHandle: calloutPointer.
  value a: 4; b: 2.

  callbackPointer := self visitChildrenClientData: calloutPointer.
  self assert: calloutPointer  equals: callbackPointer .
  value := LiblangTestStruct fromHandle: callbackPointer.
  self assert: {value a . value b} equals: #(4 2).
Libclang >> clang_visitChildren__parentCursor: parent
                      visitorCallback: visitor
                       clientData: client_data
	^ self ffiCall: #( CXChildVisitResult clang_visitChildren(
                         CXCursor parent,
                         CXCursorVisitor visitor,
                         void *  client_data))

So I’ve made some progress.  But maybe there is a better way so that client_data in the callout can be properly typed CXClientData?

Up to the start of this page is saved here…

    location: ''
    user: ''
    password: ''



This entry was posted in FFI, Pharo. Bookmark the permalink.

Leave a Reply