Headless Analyzer

Ghidra's Headless Analyzer allows users to run Ghidra from the command line without invoking the user interface.
Can be run on one file or a directory of files (including subdirectories).
Can either import programs to a project or process programs in an existing project.
Any Ghidra script including those that invoke the GUI can be run in headless mode.

How to Run the Headless Analyzer

ghidra_<version>/support/analyzeHeadless.bat|sh

  analyzeHeadless [<project_location> <project_name>[/<folder_path>]] |
    [ghidra://<server>[:<port>]/<repository_name>[/<folder_path>]]
       [[-import [<directory>|<file>]+] | [-process [<project_file>]]]
       [-preScript <ScriptName> [<arg>]*]
       [-postScript <ScriptName> [<arg>]*]
       [-scriptPath "<path1>[;<path2>...]"]
       [-propertiesPath "<path1>[;<path2>...]"]
       [-log <path to log file>] [-scriptlog <path to script log file>]
       [-overwrite] [-recursive] [-readOnly] [-deleteProject]
       [-noanalysis]
       [-processor <languageID>] [-cspec <compilerSpecID>]
       [-analysisTimeoutPerFile <timeout in seconds>]
       [-keystore <KeystorePath>] [-connect [<userID>]]
       [-p] [-commit ["<comment>"]] [-okToDelete]
       [-max-cpu <max cpu cores to use>] [-loader <desired loader name>]

NOTE: The analyzeHeadlessREADME.html has many more details and examples.

<project_name>[/<folder_path>]

The name of either an existing project or new project you wish to create in the <project_location> directory.
If the optional folder path is included, import(s) will be rooted under this project folder.

In the normal case, if you don't have an existing project of the name "project_name", then one will be created by the Headless Analyzer.

In the server case, a project must already exist.

ghidra://<server>[:<port>]/
<repository_name>[/<folder_path>]

Used to specify a Ghidra Server repository URL and folder path instead of a local project_location and project_name.
If the optional folder path is used, imports will be rooted under this directory (subfolders will be created if they don't already exist).
The named repository must already exist on the Ghidra Server.

In the normal case, if you don't have an existing project of the name "project_name", then one will be created by the Headless Analyzer.

In the server case, a project must already exist.

-import [<directory>|<file>]+

Specifies one or more executables (or directories of executables) to import.
May use wildcard characters to specify a directory or file (valid characters and behavior subject to the host OS interpretation of wildcards).
This option may be repeated to specify additional imports.
NOTE: -import and -process can not both be present in the parameters list.

If importing a directory:

Creates directory folder in project if it doesn't already exist.
When using -recursive, imports files in subdirectories as well; directory structure will be recreated in the Ghidra project.

Wildcard characters can be used to specify multiple files (i.e., "-import /home/files/n*"). Valid wildcard characters are determined by the host OS. For example:

Linux:

Allows * and ?
Allows ranges of characters: [a-z] or [!a-z]
Wildcards can expand to either directories or files

Windows:

Allows * and ?
Wildcards can only expand to files

Can specify multiple imports two ways:

Space-separated paths:

-import [directory or file] [directory or file] [directory or file]

Repeating the "-import" parameter

-import [directory or file] –import [directory or file]

-process [<project_file>]

Runs scripts and/or analysis on existing project file(s).
When [project_file] is included, it specifies existing project executable(s).
When [project_file] is omitted, runs on all the files in the project <folder_path>.
May use '*' and '?' wildcards (wildcard string must be surrounded by single quotes)
Unlike -import, -process may only be used once.

[project_file] specifies a file only (can not contain a path to a file). Use <folder_path> in the "<project_name>[/<folder_path>]" specification to identify the path to the file.

Wildcard characters are limited to "?" and "*", and strings should be surrounded by single quotes (to prevent host OS from prematurely expanding the wildcard before it is passed to the Headless Analyzer).

Can use -recursive to run scripts/analysis over subfolders of a directory.

Using -recursive with wildcard string searches subfolders for files that match the wildcard string.

Examples:

Find all .exe files starting with 'a' in the root directory

Find all .exe files starting with 'a' in the root directory and child dirs

-preScript <ScriptName.ext> [<arg>]*

Identifies the name of a script (including file extension, such as MyScript.java) that will execute before analysis. No path is necessary (script locations are specified using -scriptPath).
Parameters to the script may be passed to the script.
This option must be repeated to specify additional pre-scripts.

Headless Analyzer will look for script in default script directories and those specified by -scriptPath

-postScript <ScriptName> [<arg>]*

Identifies the name of a script (including file extension, such as MyScript.java) that will execute after analysis. No path is necessary (script locations are specified using -scriptPath).
Parameters to the script may be passed to the script.
This option must be repeated to specify additional post-scripts.

Headless Analyzer will look for script in default script directories and those specified by -scriptPath

-scriptPath "<path1>[;<path2>...]"

Specifies script search path(s).
A path may start with $GHIDRA_SCRIPT (the Ghidra installation directory) or $USER_HOME (the user's home directory).

On Unix systems, these home variables must be escaped with a '\' character.

Examples:

Windows:

-scriptPath "$GHIDRA_HOME/Ghidra/Features/Base/ghidra_scripts;/myscripts"

Unix:

-scriptPath "\$GHIDRA_HOME/Ghidra/Features/Base/ghidra_scripts;/myscripts"

Specifies script paths for all these types of scripts: Pre, post, primary, and secondary

-propertiesPath "<path1>[;<path2>...]"

Specifies *.properties file search path(s).
.properties files are used to pass parameters to scripts
A path may start with $GHIDRA_SCRIPT (the Ghidra installation directory) or $USER_HOME (the user's home directory).
NOTE: .properties files are a legacy way to pass parameters to scripts. It is now preferred to pass arguments as mentioned on previous slides.

More detail in later slide and analyzeHeadlessREADME.html file.

-log <path to log file>

Sets the location of the file that stores logging output from importing, processing, and analysis.
If not used, logging output is written to the "application.log" file in the user directory.

Captures general-purpose logging for Ghidra. Script outputs can be redirected using the "-scriptlog" parameter.

-scriptlog <path to script log file>

Sets the location of the file that stores logging output from pre- and post-scripts.
If not used, script logging output is written to the "script.log" file in the user directory.

Captures script logging for Ghidra.

-overwrite

Overwrites existing project files that conflict with an import file.
If not used, conflicting import files are skipped.

Allows an existing project file to be overwritten if it shares the same name as the imported binary.

-recursive

Enables recursive descent into directories and project sub-folders when a directory has been specified in -import or -process mode.
If not used, only the immediate files contained within the specified directory are imported or processed.

Affects -import or -process modes – used for determining whether to process files in current directory or files in current directory + all its subfolders.

-readOnly

In -import mode, imported files are not saved to the project after scripts and/or analysis.
In -process mode, any changes made to files by scripts or analysis are discarded.
The -overwrite option will be ignored if this option is specified during import operations.

For now, -readOnly must be used with -process when operating on a shared project.

-processor <languageID>

Sets the processor information to be used in -import mode (and any subsequent analysis).
If not used, Ghidra uses header info (if available) to determine the processor.
languageID can be found in the
..ghidra_<version>\Ghidra\Processors\proc_name\data\languages\proc_name.ldefs file

Example (path for x86 processor):
..ghidra_<version>\Ghidra\Processors\x86\data\languages\x86.ldefs

If Ghidra recognizes processor it uses the recognized one, if no processor recognized it uses the one specified by -processor option.

-cspec <compilerSpecID>

Sets the compiler specification to be used in -import mode (and any subsequent analysis).
'compilerSpecID' can be found in the
..ghidra_<version>\Ghidra\Processors\proc_name\data\languages\proc_name.ldefs file

If Ghidra recognizes processor it uses the preferred or default cspec for the recognized processor spec, if no processor recognized it uses the one specified by combination –processor and –spec options

-analysisTimeoutPerFile
<timeout in seconds>

Sets a timeout value, in seconds, for analysis
If analysis on a file exceeds the specified time, analysis is interrupted and processing continues as scheduled
Can use postScripts to detect if analysis has timed out (see analysisHeadlessREADME.html)

-keystore <KeystorePath>

When using a Ghidra Server with PKI or SSH authentication, allows specification of a suitable private keystore file. The file should rely on file system protection only to avoid prompting for a password.

Better if you do it in the tool first to get it set up.

-connect [<userID>]

Allows the process owner's default userID to be overridden with the given <userID> when connecting to a Ghidra Server (provided the server has been configured to allow this).

You can configure the Ghidra server to specify a different user ID. Ex: people who log in locally as root – wanted way to specify different users on Ghidra server

If you use the URL project path this option is implied – you don’t need to use this option unless you want to change the user name

-p

When connecting to a server, allows interactive prompting for a password via the console.

NOTE: In rare cases, password text will be echoed to the console (warning will show at password prompt).

Used when connecting to a server that needs a password.

This method of authentication is normally discouraged – but if not used, the server connection will likely fail authentication if a password is required.

In some cases, password text will be echoed back (NOT masked) when the user is typing (there will be a warning). Password masking seems to work when using the Ghidra jarFile, but not when using analyzeHeadless.bat/sh or eclipse launcher.

-commit ["<comment>"]

Commits all imported/processed files in a shared project to the project's underlying repository.
The <comment> is optional and is saved with all commits in the headless session.
Commits do not apply when the -readOnly parameter has been enabled.

If you use the URL project path, this option is implied – you don’t need to use this option unless you want to commit a comment when you commit.

Commits are currently not allowed for -process mode on shared projects (which must run in -readOnly mode).

-okToDelete

When running in headless mode, some scripts exist that mark programs for deletion. This deletes all versions of a program in shared project mode and cannot be undone
Therefore, so that programs are not accidentally deleted, this option is required to be set if user wants program deleted in headless mode.

-loader <desired loader name>

Forces the file to be imported using a specific loader.
Loaders can take additional arguments that they apply during the import process for things like block name, base address, file offset, length, etc... . See analyzeHeadlessREADME.html for full list.

General Notes

Make sure the specified project is not already open in the Ghidra GUI.
When importing in bulk (i.e., specifying a directory or wildcard string), files starting with '.' are ignored.

Import can be forced by naming the file during import:

-import /Users/user/.hidden.exe

Shared Project Notes

Avoid using the script API method 'setServerCredentials'.
Specify -connect with userID to override default userID (subject to server settings).
If server/keystore requires password, -connect will force prompting via stdin.
It is not possible to create a shared project via command line. See the GhidraProject class for a simple API that allows shared project creation using a script.

Why should setServerCredentials be avoided:

Because the headless analyzer already does it for you; when you use setServerCredentials, the two actions conflict with each other

Note: be careful with password – it will show up in the clear (except when using ghidra.jar).

Exercise 1

Create a directory in your home directory called "MyPrograms"
Copy some of the class exercise programs to the "MyPrograms" directory
Run Ghidra in headless so that it:

Analyzes the programs in the "MyPrograms" directory
Creates a project called "<your_name>_HeadlessProj" in your home directory that is saved after the script finishes.
Writes script output to a log called "myLog" in your home directory.
Calls the ReportPercentDisassembled.java script on each program after analysis is finished.

After the script finishes, verify that it created a project and log in the correct directory. Open the log and verify that the log contains lines that start with "REPORT DISASSEMBLY..." for each program in your "MyPrograms" directory. Open the project to verify that it contains analyzed programs from your "MyPrograms" directory.

Headless Scripting Capabilities

Can do just about anything you can do in normal scripts including GUI-dependent actions
Can make scripts behave correctly in both GUI and headless environment
Any number Ghidra scripts can be run before and/or after analysis.
For multiple pre/post-scripts, scripts are executed in the order specified on the command line.

When running scripts without -import or -process, only the specified pre/post-scripts will be executed. In this case, all scripts must execute in a program-independent manner, or errors will occur. Use -process for scripts that are program-dependent.

Headless Scripting Capabilities

Selections made in scripts will carry through to following scripts unless changed/cleared.
When running in -import mode, any pre- or post-script may invoke the setTemporary() method on currentProgram to prevent that import from being saved.
Advanced control for deleting programs or canceling future scripts is discussed in the "Control Follow-On Program Processing" section.

Why setTemporary() is useful: if you are using scripts to determine which binaries are interesting, you can save the interesting ones in your project and ditch the others.

Headless Scripting Capabilities

Some useful features...

Run other scripts from a script
Get current and default analysis options
Set analysis options

Also reset to default

Create and use selections
Detect auto-analysis timeout
Pass parameter values to a script
Control follow-on processing of a program after the current script (whether to abort future processing and/or delete program from project).

New in 6.0: build-in methods for getting/setting analysis options.

Get Analyzer Information

  
  // Print analyzer names, their current values,
  // and whether set to default value
  
  
  Map<String, String> analyzers =
      getCurrentAnalysisOptionsAndValues(currentProgram);

  for (String analyzer : analyzers.keySet()) {
    String analyzerValue = analyzers.get(analyzer);
    boolean isDefault =
        isAnalysisOptionDefaultValue(currentProgram, analyzer,
          analyzerValue);
	println(analyzer + " : " + analyzerValue +
          (isDefault ? " (default)" : ""));
  }

There are also built-in functions to:

get option choices (for those analyzers that can only be set to certain values)
get descriptions for options (as provided by the analyzer)
reset some/all analysis options to default
get default values for analysis options

Set Analysis Options

  
  // Turn off Stack Analyzer
  
  setAnalysisOption(currentProgram, "Stack", "false");


  // Turn off Stack Analyzer,
  // Turn on Decompiler Param ID

  Map<String,String> optionsToSet =
           new HashMap<String, String>();
  optionsToSet.put("Stack", "false");
  optionsToSet.put("Decompiler Parameter ID", "true");
  setAnalysisOptions(currentProgram, optionsToSet);

Note that option names and values must be strings (code will attempt to convert the value to the correct type).

Reset Analysis Options

  
  resetAllAnalysisOptions(currentProgram);


  List<String> opts = Arrays.asList(new String[] {
	"Stack", "Decompiler Parameter ID"});

  resetAnalysisOption(currentProgram, opts);

Create a Selection

  
  MemoryBlock block =
     currentProgram.getMemory().getBlock(".data");

  AddressSet set = createAddressSet();

  set.addRange(block.getStart(),block.getEnd());

  createSelection(set);

Control Follow-On Program
Processing from Scripts

Options (behavior after current script completes):


  CONTINUE
  // (default) Continue scheduled analysis/scripts,
  // save program

  CONTINUE_THEN_DELETE
  // Continue scheduled analysis/scripts, do not
  // import/save program

  ABORT
  // Abort scheduled analysis/scripts,
  // save program

  ABORT_AND_DELETE
  // Abort scheduled analysis/scripts,
  // do not import/save program

Control Follow-On Program
Processing from Scripts

Example:

  
  // Abort processing for the current file
  // because its version number is older than 8.1
  
  GhidraState currState = getState();

  if ( !getGhidraVersion().startsWith("8.1") ) {
      currState.addEnvironmentVar(
          SCRIPT_SET_CONTINUATION_STATUS,
          HeadlessContinuationOption.ABORT);
  }

More specifics on usage in analyzeHeadlessREADME.html

How to write a Headless Script

Exactly the same as writing any Ghidra script!
Best way: Use Eclipse Framework + GhidraDev plugin
GhidraDev plugin provides Headless Ghidra run configuration (requires customization of program arguments)
See ghidra_<version>/Extensions/Eclipse/GhidraDev/GhidraDev_README.html for more info.

There are two ways to install the GhidraDev plugin into your Eclipse:

Link Ghidra with an Eclipse in Front-End tool options and edit a script from the Ghidra script manager. Ghidra will offer to install GhidraDev for you if not present.

Install directly into Eclipse using the file found at ghidra_<version>/Extensions/Eclipse/GhidraDev/GhidraDev-x.x.x.zip

See the GhidraDev_README.html for pros and cons of each.