diff --git a/DevGuide.md b/DevGuide.md index 05c6a71f3b..b4109805d0 100644 --- a/DevGuide.md +++ b/DevGuide.md @@ -62,6 +62,11 @@ Build Javadoc: gradle createJavadocs ``` +Build Python3 packages for the Debugger: +``` +gradle buildPyPackage +``` + Build Ghidra to `build/dist` in an uncompressed form. This will be a distribution intended only to run on the platform on which it was built. ``` @@ -182,13 +187,18 @@ If you'd like some details of our fine tuning, take a look at [building_fid.txt] ## Debugger Development +We have recently changed the Debugger's back-end architecture. +We no longer user JNA to access native Debugger APIs. +We only use it for pseudo-terminal access. +Instead, we use Python3 and a protobuf-based TCP connection for back-end integration. + ### Additional Dependencies In addition to Ghidra's normal dependencies, you may want the following: * WinDbg for Windows x64 - * GDB 8.0 or later for Linux amd64/x86_64 - * LLDB 13.0 for macOS + * GDB 13 or later for Linux + * LLDB 10 or later for macOS The others (e.g., JNA) are handled by Gradle via Maven Central. @@ -199,121 +209,137 @@ These all currently reside in the `Ghidra/Debug` directory, but will likely be r `Framework` and `Feature` directories later. Each project is listed "bottom up" with a brief description and status. - * ProposedUtils - a collection of utilities proposed to be moved to other respective projects - * AnnotationValidator - an experimental annotation processor for database access objects + * ProposedUtils - a collection of utilities proposed to be moved to other respective projects. + * AnnotationValidator - an experimental annotation processor for database access objects. * Framework-TraceModeling - a database schema and set of interfaces for storing machine state over - time + time. * Framework-AsyncComm - a collection of utilities for asynchronous communication (packet formats and completable-future conveniences). * Framework-Debugging - specifies interfaces for debugger models and provides implementation - conveniences. + conveniences. This is mostly deprecated. * Debugger - the collection of Ghidra plugins and services comprising the Debugger UI. + * Debugger-rmi-trace - the wire protocol, client, services, and UI components for Trace RMI, the new back-end architecture. * Debugger-agent-dbgeng - the connector for WinDbg (via dbgeng.dll) on Windows x64. * Debugger-agent-dbgmodel - an experimental connector for WinDbg Preview (with TTD, via - dbgmodel.dll) on Windows x64. - * Debugger-agent-dbgmodel-traceloader - an experimental "importer" for WinDbg trace files. - * Debugger-agent-gdb - the connector for GDB (8.0 or later recommended) on UNIX. - * Debugger-swig-lldb - the Java language bindings for LLDB's SBDebugger, also proposed upstream. - * Debugger-agent-lldb - the connector for LLDB (13.0 required) on macOS, UNIX, and Windows. + dbgmodel.dll) on Windows x64. This is deprecated, as most of these features are implemented in Debugger-agent-dbgeng for the new architecture. + * Debugger-agent-dbgmodel-traceloader - an experimental "importer" for WinDbg trace files. This is deprecated. + * Debugger-agent-gdb - the connector for GDB (13 or later recommended) on UNIX. + * Debugger-swig-lldb - the Java language bindings for LLDB's SBDebugger, also proposed upstream. This is deprecated. We now use the Python3 language bindings for LLDB. + * Debugger-agent-lldb - the connector for LLDB (10 or later recommended) on macOS, UNIX, and Windows. * Debugger-gadp - the connector for our custom wire protocol the Ghidra Asynchronous Debugging - Protocol. - * Debugger-jpda - an in-development connector for Java and Dalvik debugging via JDI (i.e., JDWP). + Protocol. This is deprecated. It's replaced by Debugger-rmi-trace. + * Debugger-jpda - an in-development connector for Java and Dalvik debugging via JDI (i.e., JDWP). This is deprecated and not yet replaced. The Trace Modeling schema records machine state and markup over time. -It rests on the same database framework as Programs, allowing trace recordings to be stored in a -Ghidra project and shared via a server, if desired. Trace "recording" is a de facto requirement for -displaying information in Ghidra's UI. However, only the machine state actually observed by the user -(or perhaps a script) is recorded. For most use cases, the Trace is small and ephemeral, serving -only to mediate between the UI components and the target's model. It supports many of the same -markup (e.g., disassembly, data types) as Programs, in addition to tracking active threads, loaded -modues, breakpoints, etc. +It rests on the same database framework as Programs, allowing trace recordings to be stored in a Ghidra project and shared via a server, if desired. +Trace "recording" is a de facto requirement for displaying information in Ghidra's UI. +The back-end connector has full discretion over what is recorded by using Trace RMI. +Typically, only the machine state actually observed by the user (or perhaps a script) is recorded. +For most use cases, the Trace is small and ephemeral, serving only to mediate between the UI components and the target's model. +It supports many of the same markup (e.g., disassembly, data types) as Programs, in addition to tracking active threads, loaded modues, breakpoints, etc. -Every model (or "adapter" or "connector" or "agent") implements the API specified in -Framework-Debugging. As a general rule in Ghidra, no component is allowed to access a native API and -reside in the same JVM as the Ghidra UI. This allows us to contain crashes, preventing data loss. To -accommodate this requirement -- given that debugging native applications is almost certainly going -to require access to native APIs -- we've developed the Ghidra Asynchronous Debugging Protocol. This -protocol is tightly coupled to Framework-Debugging, essentially exposing its methods via RMI. The -protocol is built using Google's Protobuf library, providing a potential path for agent -implementations in alternative languages. GADP provides both a server and a client implementation. -The server can accept any model which adheres to the specification and expose it via TCP; the client -does the converse. When a model is instantiated in this way, it is called an "agent," because it is -executing in its own JVM. The other connectors, which do not use native APIs, may reside in Ghidra's -JVM and typically implement alternative wire protocols, e.g., JDWP. In both cases, the -implementations inherit from the same interfaces. +Every back end (or "adapter" or "connector" or "agent") employs the Trace RMI client to populate a trace database. +As a general rule in Ghidra, no component is allowed to access a native API and reside in the same JVM as the Ghidra UI. +This allows us to contain crashes, preventing data loss. +To accommodate this requirement — given that debugging native applications is almost certainly going to require access to native APIs — we've developed the Trace RMI protocol. +This also allows us to better bridge the language gap between Java and Python, which is supported by most native debuggers. +This protocol is loosely coupled to Framework-TraceModeling, essentially exposing its methods via RMI, as well as some methods for controlling the UI. +The protocol is built using Google's Protobuf library, providing a potential path for back-end implementations in alternative languages. +We provide the Trace RMI server as a Ghidra component implemented in Java and the Trace RMI client as a Python3 package. +A back-end implementation may be a stand-alone executable or script that accesses the native debugger's API, or a script or plugin for the native debugger. +It then connects to Ghidra via Trace RMI to populate the trace database with information gleaned from that API. +It should provide a set of diagnostic commands to control and monitor that connection. +It should also use the native API to detect session and target changes so that Ghidra's UI consistently reflects the debugging session. -The Debugger services maintain a collection of active connections and inspect each model for -potential targets. When a target is found, the service inspects the target environment and attempts -to find a suitable opinion. Such an opinion, if found, instructs Ghidra how to map the objects, -addresses, registers, etc. from the target namespace into Ghidra's. The target is then handed to a -Trace Recorder which begins collecting information needed to populate the UI, e.g., the program -counter, stack pointer, and the bytes of memory they refer to. +The old system relied on a "recorder" to discover targets and map them to traces in the proper Ghidra language. +That responsibility is now delegated to the back end. +Typically, it examines the target's architecture and immediately creates a trace upon connection. ### Developing a new connector So Ghidra does not yet support your favorite debugger? -It is tempting, exciting, but also daunting to develop your own connector. -Please finish reading this guide, and look carefully at the ones we have so far, and perhaps ask to -see if we are already developing one. Of course, in time you might also search the internet to see -if others are developing one. There are quite a few caveats and gotchas, the most notable being that -this interface is still in quite a bit of flux. When things go wrong, it could be because of, -without limitation: 1) a bug on your part, 2) a bug on our part, 3) a design flaw in the interfaces, -or 4) a bug in the debugger/API you're adapting. We are still in the process of writing up this -documentation. In the meantime, we recommend using the GDB and dbgeng.dll agents as examples. +We believe the new system is much less daunting than the previous. +Still, please finish reading this guide, and look carefully at the ones we have so far, and perhaps ask to see if we are already developing one. +Of course, in time you might also search the internet to see if others are developing one. +There are quite a few caveats and gotchas, the most notable being that this interface is still in some flux. +When things go wrong, it could be because of, without limitation: -You'll also need to provide launcher(s) so that Ghidra knows how to configure and start your -connector. Please provide launchers for your model in both configurations: as a connector in -Ghidra's JVM, and as a GADP agent. If your model requires native API access, you should only permit -launching it as a GADP agent, unless you give ample warning in the launcher's description. Look at -the existing launchers for examples. There are many model implementation requirements that cannot be -expressed in Java interfaces. Failing to adhere to those requirements may cause different behaviors -with and without GADP. Testing with GADP tends to reveal those implementation errors, but also -obscures the source of client method calls behind network messages. We've also codified (or -attempted to codify) these requirements in a suite of abstract test cases. See the `ghidra.dbg.test` -package of Framework-Debugging, and again, look at existing implementations. +1. A bug on your part +2. A bug on our part +3. A design flaw in the interfaces +4. A bug in the debugger/API you're adapting + +We are still (yes, still) in the process of writing up this documentation. +In the meantime, we recommend using the GDB and dbgeng agents as examples. +Be sure to look at the Python code `src/main/py`! +The deprecated Java code `src/main/java` is still included as we transition. + +You'll also need to provide launcher(s) so that Ghidra knows how to configure and start your connector. +These are just shell scripts. +We use bash scripts on Linux and macOS, and we use batch files on Windows. +Try to include as many common use cases as makes sense for the debugger. +This provides the most flexibility to users and examples to power users who might create derivative launchers. +Look at the existing launchers for examples. + +For testing, please follow the examples for GDB. +We no longer provide abstract classes that prescribe requirements. +Instead, we just provide GDB as an example. +Usually, we split our tests into three categories: + + * Commands + * Methods + * Hooks + +The Commands tests check that the user CLI commands, conventionally implemented in `commands.py`, work correctly. +In general, do the minimum connection setup, execute the command, and check that it produces the expected output and causes the expected effects. + +The Methods tests check that the remote methods, conventionally implemented in `methods.py`, work correctly. +Many methods are just wrappers around CLI commands, some provided by the native debugger and some provided by `commands.py`. +These work similarly to the commands test, except that they invoke methods instead of executing commands. +Again, check the return value (rarely applicable) and that it causes the expected effects. + +The Hooks tests check that the back end is able to listen for session and target changes, e.g., knowing when the target stops. +*The test should not "cheat" by executing commands or invoking methods that should instead be triggered by the listener.* +It should execute the minimal commands to setup the test, then trigger an event. +It should then check that the event in turn triggered the expected effects, e.g., updating PC upon the target stopping. + +Whenever you make a change to the Python code, you'll need to re-assemble the package's source. + +``` +gradle assemblePyPackage +``` + +This is required in case your package includes generated source, as is the case for Debugger-rmi-trace. +If you want to create a new Ghidra module for your connector (recommended) use an existing one's `build.gradle` as a template. +A key part is applying the `hasPythonPackage.gradle` script. ### Adding a new platform -If an existing connector exists for a suitable debugger on the desired platform, then adding it may -be very simple. For example, both the x86 and ARM platforms are supported by GDB, so even though -we're currently focused on x86 support, we've provided the opinions needed for Ghidra to debug ARM -platforms (and several others) via GDB. These opinions are kept in the "Debugger" project, not their -respective "agent" projects. We imagine there are a number of platforms that could be supported -almost out of the box, except that we haven't written the necessary opinions, yet. Take a look at -the existing ones for examples. +If a connector already exists for a suitable debugger on the desired platform, then adding it may be very simple. +For example, many platforms are supported by GDB, so even though we're currently focused on x86-64 (and to some extent arm64) support, we've provided the mappings for many. +These mappings are conventionally kept in each connector's `arch.py` file. -In general, to write a new opinion, you need to know: 1) What the platform is called (including -variant names) by the debugger, 2) What the processor language is called by Ghidra, 3) If -applicable, the mapping of target address spaces into Ghidra's address spaces, 4) If applicable, the -mapping of target register names to those in Ghidra's processor language. In most cases (3) and (4) -are already implemented by default mappers, so you can use those same mappers in your opinion. Once -you have the opinion written, you can try debugging and recording a target. If Ghidra finds your -opinion applicable to that target, it will attempt to record, and then you can work out the kinds -from there. Again, we have a bit of documentation to do regarding common pitfalls. +In general, to update `arch.py`, you need to know: + +1. What the platform is called (including variant names) by the debugger +2. What the processor language is called by Ghidra +3. If applicable, the mapping of target address spaces into Ghidra's address spaces +4. If applicable, the mapping of target register names to those in Ghidra's processor language + +In most cases (3) and (4) are already implemented by the included mappers. +Naturally, you'll want to test the special cases, preferably in automated tests. ### Emulation -The most obvious integration path for 3rd-party emulators is to write a "connector." However, p-code -emulation is now an integral feature of the Ghidra UI, and it has a fairly accessible API. Namely, -for interpolation between machines states recorded in a trace, and extrapolation into future machine -states. Integration of such emulators may still be useful to you, but we recommend trying the p-code -emulator to see if it suits your needs for emulation in Ghidra before pursuing integration of -another emulator. +The most obvious integration path for 3rd-party emulators is to write a "connector." +However, p-code emulation is an integral feature of the Ghidra UI, and it has a fairly accessible API. +Namely, for interpolation between machines states recorded in a trace, and extrapolation into future machine states. +Integration of such emulators may still be useful to you, but we recommend trying the p-code emulator to see if it suits your needs for emulation in Ghidra before pursuing integration of another emulator. +We also provide out-of-the-box QEMU integration via GDB. ### Contributing -Whether submitting help tickets and pull requests, please tag those related to the debugger with -"Debugger" so that we can triage them more quickly. - -To set up your environment, in addition to the usual Gradle tasks, process the Protobuf -specification for GADP: - -```bash -gradle generateProto -``` - -If you already have an environment set up in Eclipse, please re-run `gradle prepDev eclipse` and -import the new projects. +When submitting help tickets and pull requests, please tag those related to the debugger with "Debugger" so that we can triage them more quickly. [java]: https://dev.java diff --git a/Ghidra/Debug/Debugger-agent-dbgeng/Module.manifest b/Ghidra/Debug/Debugger-agent-dbgeng/Module.manifest index e69de29bb2..d81b91c24e 100644 --- a/Ghidra/Debug/Debugger-agent-dbgeng/Module.manifest +++ b/Ghidra/Debug/Debugger-agent-dbgeng/Module.manifest @@ -0,0 +1,5 @@ +MODULE FILE LICENSE: pypkg/dist/capstone-5.0.1-py3-none-win_amd64.whl BSD-3-CAPSTONE +MODULE FILE LICENSE: pypkg/dist/comtypes-1.4.1-py3-none-any.whl MIT +MODULE FILE LICENSE: pypkg/dist/Pybag-2.2.10-py3-none-any.whl MIT +MODULE FILE LICENSE: pypkg/dist/pywin32-306-cp312-cp312-win_amd64.whl Python Software Foundation License + diff --git a/Ghidra/Debug/Debugger-agent-dbgeng/build.gradle b/Ghidra/Debug/Debugger-agent-dbgeng/build.gradle index 8ea79a8ce0..6299b9b1b8 100644 --- a/Ghidra/Debug/Debugger-agent-dbgeng/build.gradle +++ b/Ghidra/Debug/Debugger-agent-dbgeng/build.gradle @@ -77,6 +77,11 @@ else { } } +distributePyDep("Pybag-2.2.10-py3-none-any.whl") +distributePyDep("capstone-5.0.1-py3-none-win_amd64.whl") +distributePyDep("comtypes-1.4.1-py3-none-any.whl") +distributePyDep("pywin32-306-cp312-cp312-win_amd64.whl") + tasks.nodepJar { manifest { attributes['Main-Class'] = 'agent.dbgeng.gadp.DbgEngGadpServer' diff --git a/Ghidra/Debug/Debugger-agent-dbgeng/certification.manifest b/Ghidra/Debug/Debugger-agent-dbgeng/certification.manifest index 497d451482..cca8607dd9 100644 --- a/Ghidra/Debug/Debugger-agent-dbgeng/certification.manifest +++ b/Ghidra/Debug/Debugger-agent-dbgeng/certification.manifest @@ -1,5 +1,6 @@ ##VERSION: 2.0 ##MODULE IP: Apache License 2.0 +##MODULE IP: MIT Module.manifest||GHIDRA||||END| data/debugger-launchers/local-dbgeng.bat||GHIDRA||||END| data/debugger-launchers/local-ttd.bat||GHIDRA||||END| diff --git a/Ghidra/Debug/Debugger-agent-dbgeng/src/main/py/README.md b/Ghidra/Debug/Debugger-agent-dbgeng/src/main/py/README.md index 2a84727524..2b7f1ec084 100644 --- a/Ghidra/Debug/Debugger-agent-dbgeng/src/main/py/README.md +++ b/Ghidra/Debug/Debugger-agent-dbgeng/src/main/py/README.md @@ -1,8 +1,3 @@ # Ghidra Trace RMI Package for connecting dbgeng to Ghidra via Trace RMI. - -This connector requires Pybag 2.2.8 or better: - - https://pypi.org/project/Pybag - https://github.com/dshikashio/Pybag \ No newline at end of file diff --git a/Ghidra/Debug/Debugger-agent-dbgeng/src/main/py/pyproject.toml b/Ghidra/Debug/Debugger-agent-dbgeng/src/main/py/pyproject.toml index 77f367e634..a07be47666 100644 --- a/Ghidra/Debug/Debugger-agent-dbgeng/src/main/py/pyproject.toml +++ b/Ghidra/Debug/Debugger-agent-dbgeng/src/main/py/pyproject.toml @@ -1,6 +1,6 @@ [build-system] -requires = ["hatchling"] -build-backend = "hatchling.build" +requires = ["setuptools"] +build-backend = "setuptools.build_meta" [project] name = "ghidradbg" diff --git a/Ghidra/Debug/Debugger-agent-gdb/src/main/py/pyproject.toml b/Ghidra/Debug/Debugger-agent-gdb/src/main/py/pyproject.toml index cbff2cad29..f9c3c4a20d 100644 --- a/Ghidra/Debug/Debugger-agent-gdb/src/main/py/pyproject.toml +++ b/Ghidra/Debug/Debugger-agent-gdb/src/main/py/pyproject.toml @@ -1,6 +1,6 @@ [build-system] -requires = ["hatchling"] -build-backend = "hatchling.build" +requires = ["setuptools"] +build-backend = "setuptools.build_meta" [project] name = "ghidragdb" diff --git a/Ghidra/Debug/Debugger-agent-lldb/src/main/py/pyproject.toml b/Ghidra/Debug/Debugger-agent-lldb/src/main/py/pyproject.toml index df65c6dee9..7c279cf07a 100644 --- a/Ghidra/Debug/Debugger-agent-lldb/src/main/py/pyproject.toml +++ b/Ghidra/Debug/Debugger-agent-lldb/src/main/py/pyproject.toml @@ -1,6 +1,6 @@ [build-system] -requires = ["hatchling"] -build-backend = "hatchling.build" +requires = ["setuptools"] +build-backend = "setuptools.build_meta" [project] name = "ghidralldb" diff --git a/Ghidra/Debug/Debugger-rmi-trace/Module.manifest b/Ghidra/Debug/Debugger-rmi-trace/Module.manifest index e69de29bb2..fefac7bf9c 100644 --- a/Ghidra/Debug/Debugger-rmi-trace/Module.manifest +++ b/Ghidra/Debug/Debugger-rmi-trace/Module.manifest @@ -0,0 +1,4 @@ +MODULE FILE LICENSE: pypkg/dist/protobuf-3.20.3-py2.py3-none-any.whl BSD-3-GOOGLE +MODULE FILE LICENSE: pypkg/dist/psutil-5.9.8.tar.gz BSD-3-PSUTIL +MODULE FILE LICENSE: pypkg/dist/setuptools-68.0.0-py3-none-any.whl MIT +MODULE FILE LICENSE: pypkg/dist/wheel-0.37.1-py2.py3-none-any.whl MIT diff --git a/Ghidra/Debug/Debugger-rmi-trace/build.gradle b/Ghidra/Debug/Debugger-rmi-trace/build.gradle index 2c9cf5fc22..90cfde5a91 100644 --- a/Ghidra/Debug/Debugger-rmi-trace/build.gradle +++ b/Ghidra/Debug/Debugger-rmi-trace/build.gradle @@ -63,3 +63,8 @@ tasks.assemblePyPackage { into "src/ghidratrace" } } + +distributePyDep("protobuf-3.20.3-py2.py3-none-any.whl") +distributePyDep("psutil-5.9.8.tar.gz") +distributePyDep("setuptools-68.0.0-py3-none-any.whl") +distributePyDep("wheel-0.37.1-py2.py3-none-any.whl") diff --git a/Ghidra/Debug/Debugger-rmi-trace/certification.manifest b/Ghidra/Debug/Debugger-rmi-trace/certification.manifest index bdc45f52b2..e339fc31b5 100644 --- a/Ghidra/Debug/Debugger-rmi-trace/certification.manifest +++ b/Ghidra/Debug/Debugger-rmi-trace/certification.manifest @@ -1,4 +1,6 @@ ##VERSION: 2.0 +##MODULE IP: BSD-3-GOOGLE +##MODULE IP: BSD-3-PSUTIL DEVNOTES.txt||GHIDRA||||END| Module.manifest||GHIDRA||||END| data/ExtensionPoint.manifest||GHIDRA||||END| diff --git a/Ghidra/Debug/Debugger-rmi-trace/src/main/py/pyproject.toml b/Ghidra/Debug/Debugger-rmi-trace/src/main/py/pyproject.toml index ee4b7aab28..69231af532 100644 --- a/Ghidra/Debug/Debugger-rmi-trace/src/main/py/pyproject.toml +++ b/Ghidra/Debug/Debugger-rmi-trace/src/main/py/pyproject.toml @@ -1,10 +1,10 @@ [build-system] -requires = ["hatchling"] -build-backend = "hatchling.build" +requires = ["setuptools"] +build-backend = "setuptools.build_meta" [project] name = "ghidratrace" -version = "10.4" +version = "11.1" authors = [ { name="Ghidra Development Team" }, ] diff --git a/GhidraDocs/GhidraClass/Debugger/A1-GettingStarted.html b/GhidraDocs/GhidraClass/Debugger/A1-GettingStarted.html index b0b3da0890..10ce5c8023 100644 --- a/GhidraDocs/GhidraClass/Debugger/A1-GettingStarted.html +++ b/GhidraDocs/GhidraClass/Debugger/A1-GettingStarted.html @@ -327,6 +327,12 @@ tell Ghidra where it is, then in the launcher drop-down, select Configure and Launch termmines using… → gdb. DO NOT select Re-launch termmines using gdb, since this will not allow you to correct the configuration.

+

If it looks like there’s an error about importing python packages, +e.g., “google protobuf,” then you need to install some dependencies. +These are listed in the launcher’s description. For your convenience, +the correct versions are distributed with Ghidra. Search for files +ending in .whl (or .tar.gz) and install the +required ones using python3 -m pip install.

The Dynamic Listing is empty

@@ -457,8 +463,9 @@ the system’s copy of GDB. Probably, you can just click
  • In the Model window (to the left), expand the Available node.
  • In the filter box, type termmines.
    • -
  • Right click on the node and select Attach, or, if you prefer, note the PID, e.g. 1234, then in the Terminal type, -e.g., attach 1234.
    • +
  • Right click on the node and select Attach, or, if +you prefer, note the PID, e.g. 1234, then in the +Terminal type, e.g., attach 1234.
    • diff --git a/GhidraDocs/GhidraClass/Debugger/A1-GettingStarted.md b/GhidraDocs/GhidraClass/Debugger/A1-GettingStarted.md index 7616c92af6..47c6dbfb15 100644 --- a/GhidraDocs/GhidraClass/Debugger/A1-GettingStarted.md +++ b/GhidraDocs/GhidraClass/Debugger/A1-GettingStarted.md @@ -130,6 +130,11 @@ If it is just missing, then install it and try again. If you need to tell Ghidra where it is, then in the launcher drop-down, select **Configure and Launch termmines using... → gdb**. DO NOT select **Re-launch termmines using gdb**, since this will not allow you to correct the configuration. +If it looks like there's an error about importing python packages, e.g., "google protobuf," then you need to install some dependencies. +These are listed in the launcher's description. +For your convenience, the correct versions are distributed with Ghidra. +Search for files ending in `.whl` (or `.tar.gz`) and install the required ones using `python3 -m pip install`. + ### The Dynamic Listing is empty Check for an actual connection. diff --git a/GhidraDocs/GhidraClass/Debugger/B1-RemoteTargets.html b/GhidraDocs/GhidraClass/Debugger/B1-RemoteTargets.html index c4a22c0254..d7da3194cc 100644 --- a/GhidraDocs/GhidraClass/Debugger/B1-RemoteTargets.html +++ b/GhidraDocs/GhidraClass/Debugger/B1-RemoteTargets.html @@ -210,24 +210,40 @@ the target environment. Notice that we are not using gdb by forwarding Trace RMI over SSH. See the help (press F1 on the gdb via ssh menu item for advantages and disadvantages of using this -vs. gdbserver.

    +vs. gdbserver.)

    1. First, prepare the target. This is more involved than using gdbserver, since you will need to ensure gdb and the Trace RMI plugin for it are installed. The packages, which should be included with Ghidra, are ghidratrace and -ghidragdb. If you installed gdb and -python3 from your distribution’s repositories, installation -of the Python packages should just be a matter of using -pip:

      +ghidragdb, but you may need to build them first. If you +installed gdb and python3 from your +distribution’s repositories, installation of the Python packages should +be straightfoward. Search the Ghidra installation for files ending in +.whl. If the ghidratrace and +ghidragdb packages are there, you can skip this build step +and just transfer them to the target. On the local system:

      python3 -m pip install /path/to/ghidratrace....whl
      -

      Chances are, GDB embeds the same Python, so they become importable -from gdb:

      +class="sourceCode bash">python3 -m pip install build # unless you already have it +cd /path/to/ghidra/Ghidra/Debug/Debugger-rmi-trace/pypkg +python3 -m build +

      This will output .tar.gz and .whl files +under pypkg/dist. Do the same for +Debugger-agent-gdb/pypkg. Transfer the resulting +.whl files to the target, then on the target system:

      python import ghidragdb
      -

      You can quit GDB, since that was just for verifying the -installation.

      2. +class="sourceCode bash">python3 -m pip install /path/to/ghidratrace-[version].whl /path/to/ghidragdb-[version].whl +

      If you are offline, the dependencies are included in the +pypkg/dist directory for each module. Transfer and install +them, too. You may try +python -m pip install --no-index -f /path/to/packages ghidragdb, +if all the packages and dependencies are in the one directory. Chances +are, GDB embeds the same Python, so they become importable from GDB. +Test using gdb on the target system:

      +
      python import ghidragdb
      +

      No news is good news! You can quit GDB, since that was just for +verifying the installation.

    2. From the launch menu, select gdb via ssh.

      Troubleshooting

      I can’t find the Python packages to install

      -

      These should be located in the Ghidra installation. Search for files -ending in .whl. Alternatively, you can build the packages -from source. The source is included with Ghidra. If you are able to do -local debugging with Ghidra and gdb, then the source is -definitely present and functioning.

      -
      python3 -m pip install build
-cd /path/to/ghidra/Ghidra/Debug/Debugger-rmi-trace/pypkg
-python3 -m build
      -

      This should output a .whl file. Send that over to the -target system and install it. If that doesn’t work, then in the worst -case, copy the Python source over and add it to your -PYTHONPATH.

      +

      You may need to build them using the instructions above. The +dependencies are included in the Ghidra installation, but perhaps +something has gone missing. Search for files ending in .whl +or .tar.gz; they should be located in +pypkg/dist in various modules. If you are able to do local +debugging with Ghidra and gdb, then the source is +definitely present and functioning. To (re-)build the packages from +source:

      +
      python3 -m pip install build
+cd /path/to/ghidra/Ghidra/Debug/Debugger-rmi-trace/pypkg
+python3 -m build
      +

      This should output a .tar.gz and a .whl +file under pypkg/dist. Send the .whl over to +the target system and pip install it. Do the same for +Debugger-agent-gdb. If that doesn’t work, then in the worst case, copy +the Python source over and add it to your PYTHONPATH.

      The python import ghidragdb command fails

      Double-check that you have installed all the required packages and their dependencies. A common forgotten or incorrectly-versioned dependency is protobuf. We developed using -protobuf==3.20.3.

      +protobuf==3.20.3. Its “sdist” package is distributed with +Ghidra under Debugger-rmi-trace/pypkg/dist for your +convenience.

      It is also possible that gdb has embedded a different version of the interpreter than the one that python3 provides. This can happen if you built GDB or Python yourself, or you installed them from a non-standard repository. Check the actual path of the Python library used by gdb:

      -
      ldd $(which gdb)
      -

      Or, inside gdb:

      (gdb) python-interactive
->>> import sys
->>> sys.version
      +class="sourceCode bash">ldd $(which gdb) +

      Or, inside gdb:

      +
      (gdb) python-interactive
+>>> import sys
+>>> sys.version

      Suppose this identifies version 3.7. Retry the installation commands using python3.7 -m pip .... If you have multiple copies of the same version in different locations, you may need to invoke @@ -296,8 +318,8 @@ the same version in different locations, you may need to invoke

    3. First, prepare the target. This time, you will need to start gdbserver on the remote system manually. For demonstration, we will listen on 10.0.0.1 port 12345:

      -
      gdbserver 10.0.0.1:12345 termmines
      4. +
      gdbserver 10.0.0.1:12345 termmines

    4. From the launch menu, select remote gdb.

    5. Fill out the options appropriately. Notably, enter “10.0.0.1” for @@ -330,14 +352,14 @@ it.

      6. acceptor’s port number in the Connections window, e.g., “12345.”

    6. Now, on the remote system, start gdb and type:

      -
      python import ghidragdb
-file termmines
-# set args, if you'd like
-ghidra trace connect 10.0.0.1:12345
-ghidra trace start
-ghidra trace sync-enable
-starti
      7. +
      python import ghidragdb
+file termmines
+# set args, if you'd like
+ghidra trace connect 10.0.0.1:12345
+ghidra trace start
+ghidra trace sync-enable
+starti

    At this point, most things will work the same as they would for a local target. You may notice Ghidra has not given you a new terminal. diff --git a/GhidraDocs/GhidraClass/Debugger/B1-RemoteTargets.md b/GhidraDocs/GhidraClass/Debugger/B1-RemoteTargets.md index d5b0320450..0aa6155984 100644 --- a/GhidraDocs/GhidraClass/Debugger/B1-RemoteTargets.md +++ b/GhidraDocs/GhidraClass/Debugger/B1-RemoteTargets.md @@ -53,23 +53,41 @@ At this point, most things will work the same as they would for a local target. In this configuration, Ghidra will be located in the user'ls local environment, while `gdb` and the specimen will be located in the target environment. Notice that we are *not* using `gdbserver`. We will connect the local Ghidra to the remote `gdb` by forwarding Trace RMI over SSH. -See the help (press **`F1`** on the **gdb via ssh** menu item for advantages and disadvantages of using this vs. `gdbserver`. +See the help (press **`F1`** on the **gdb via ssh** menu item for advantages and disadvantages of using this vs. `gdbserver`.) 1. First, prepare the target. This is more involved than using `gdbserver`, since you will need to ensure `gdb` and the Trace RMI plugin for it are installed. - The packages, which should be included with Ghidra, are `ghidratrace` and `ghidragdb`. - If you installed `gdb` and `python3` from your distribution's repositories, installation of the Python packages should just be a matter of using `pip`: + The packages, which should be included with Ghidra, are `ghidratrace` and `ghidragdb`, but you may need to build them first. + If you installed `gdb` and `python3` from your distribution's repositories, installation of the Python packages should be straightfoward. + Search the Ghidra installation for files ending in `.whl`. + If the `ghidratrace` and `ghidragdb` packages are there, you can skip this build step and just transfer them to the target. + On the local system: ```bash - python3 -m pip install /path/to/ghidratrace....whl + python3 -m pip install build # unless you already have it + cd /path/to/ghidra/Ghidra/Debug/Debugger-rmi-trace/pypkg + python3 -m build ``` - Chances are, GDB embeds the same Python, so they become importable from `gdb`: + This will output `.tar.gz` and `.whl` files under `pypkg/dist`. + Do the same for `Debugger-agent-gdb/pypkg`. + Transfer the resulting `.whl` files to the target, then on the target system: + + ```bash + python3 -m pip install /path/to/ghidratrace-[version].whl /path/to/ghidragdb-[version].whl + ``` + + If you are offline, the dependencies are included in the `pypkg/dist` directory for each module. + Transfer and install them, too. + You may try `python -m pip install --no-index -f /path/to/packages ghidragdb`, if all the packages and dependencies are in the one directory. + Chances are, GDB embeds the same Python, so they become importable from GDB. + Test using `gdb` on the target system: ```gdb python import ghidragdb ``` + No news is good news! You can quit GDB, since that was just for verifying the installation. 1. From the launch menu, select **gdb via ssh**. @@ -87,11 +105,11 @@ At this point, most things will work the same as they would for a local target. #### I can't find the Python packages to install -These should be located in the Ghidra installation. -Search for files ending in `.whl`. -Alternatively, you can build the packages from source. -The source is included with Ghidra. +You may need to build them using the instructions above. +The dependencies are included in the Ghidra installation, but perhaps something has gone missing. +Search for files ending in `.whl` or `.tar.gz`; they should be located in `pypkg/dist` in various modules. If you are able to do local debugging with Ghidra and `gdb`, then the source is definitely present and functioning. +To (re-)build the packages from source: ```bash python3 -m pip install build @@ -99,8 +117,9 @@ cd /path/to/ghidra/Ghidra/Debug/Debugger-rmi-trace/pypkg python3 -m build ``` -This should output a `.whl` file. -Send that over to the target system and install it. +This should output a `.tar.gz` and a `.whl` file under `pypkg/dist`. +Send the `.whl` over to the target system and `pip install` it. +Do the same for Debugger-agent-gdb. If that doesn't work, then in the worst case, copy the Python source over and add it to your `PYTHONPATH`. #### The `python import ghidragdb` command fails @@ -108,6 +127,7 @@ If that doesn't work, then in the worst case, copy the Python source over and ad Double-check that you have installed all the required packages and their dependencies. A common forgotten or incorrectly-versioned dependency is `protobuf`. We developed using `protobuf==3.20.3`. +Its "sdist" package is distributed with Ghidra under `Debugger-rmi-trace/pypkg/dist` for your convenience. It is also possible that `gdb` has embedded a different version of the interpreter than the one that `python3` provides. This can happen if you built GDB or Python yourself, or you installed them from a non-standard repository. diff --git a/GhidraDocs/InstallationGuide.html b/GhidraDocs/InstallationGuide.html index 9e350bb59a..3a9b217e8e 100644 --- a/GhidraDocs/InstallationGuide.html +++ b/GhidraDocs/InstallationGuide.html @@ -32,6 +32,7 @@ future releases.

  • Ghidra Installation Directory Layout
  • Building Ghidra Native Components
    • +
  • Installing the Debuggers' Python Dependencies
  • Running Ghidra
    • +
  • Python 3.9 or later (for Debugger support)
    • +

    (Back to Top)

    @@ -343,6 +348,19 @@ the relevant modules' build/os/<platform>/ subdirectories, which wi existing pre-built native binaries in the os/<platform>/ subdirectories.

    (Back to Top)

    +

    Installing the Debugger's Python Dependencies

    +

    The Debugger now uses Python to connect to the host platform's native debuggers. This requires +Python 3.9 or later and some additional packages. These packages are included in the distribution, +but you may still install them from PyPI if you prefer:

    + +

    Different native debuggers have varying requirements, so you do not necessarily have to install +all of the above packages. Each connector will inform you of its specific requirements and where +they must be installed. In some cases, you may need to install packages on the target system.

    +

    Running Ghidra

    GUI Mode

      diff --git a/README.md b/README.md index c747fa8958..a6969c1b4b 100644 --- a/README.md +++ b/README.md @@ -48,6 +48,8 @@ To create the latest development build for your platform from this source reposi ##### Install build tools: * [JDK 17 64-bit][jdk17] * [Gradle 7.3+][gradle] +* [Python3][py3] + - [build][py3-build] module: `pip install build` * make, gcc, and g++ (Linux/macOS-only) * [Microsoft Visual Studio][vs] 2017+ or [Microsoft C++ Build Tools][vcbuildtools] with the following components installed (Windows-only): @@ -128,6 +130,8 @@ source project. [releases]: https://github.com/NationalSecurityAgency/ghidra/releases [jdk17]: https://adoptium.net/temurin/releases [gradle]: https://gradle.org/releases/ +[py3]: https://www.python.org/downloads/ +[py3-build]: https://pypi.org/project/build/ [vs]: https://visualstudio.microsoft.com/vs/community/ [vcbuildtools]: https://visualstudio.microsoft.com/visual-cpp-build-tools/ [eclipse]: https://www.eclipse.org/downloads/packages/ diff --git a/gradle/debugger/hasPythonPackage.gradle b/gradle/debugger/hasPythonPackage.gradle index 0a1afb25ce..c199cb6f0a 100644 --- a/gradle/debugger/hasPythonPackage.gradle +++ b/gradle/debugger/hasPythonPackage.gradle @@ -20,7 +20,7 @@ def checkPythonVersion(String pyCmd) { exec { commandLine pyCmd, "-c", "import sys; print('{0}.{1}'.format(*sys.version_info))" standardOutput = stdout - standardError = null + errorOutput = OutputStream.nullOutputStream() } def version = "$stdout".strip() return version @@ -33,7 +33,7 @@ def checkPythonVersion(String pyCmd) { ext.SUPPORTED_PY_VERSIONS = ['3.7', '3.8', '3.9', '3.10', '3.11', '3.12'] def findPython3() { - for (pyCmd in ['python3', 'python']) { + for (pyCmd in ['py', 'python3', 'python']) { if (checkPythonVersion(pyCmd) in SUPPORTED_PY_VERSIONS) { return pyCmd } @@ -44,6 +44,19 @@ def findPython3() { ext.PYTHON3 = findPython3() +ext.findPyDep = { name -> + File inDeps = file("${DEPS_DIR}/${project.name}/${name}") + File inRepo = file("${BIN_REPO}/${getGhidraRelativePath(project)}/${name}") + if (inDeps.exists()) { + return inDeps + } + if (inRepo.exists()) { + return inRepo + } + println("Warning: Could not find '${name}' for ${project.name}") + return inDeps +} + task assemblePyPackage(type: Copy) { from "src/main/py" into "build/pypkg/" @@ -51,25 +64,45 @@ task assemblePyPackage(type: Copy) { rootProject.tasks.prepDev.dependsOn(assemblePyPackage) -task buildPyPackage(type: Exec) { +task buildPyPackage { ext.dist = { file("build/pypkg/dist") } inputs.files(assemblePyPackage) outputs.dir(dist) - workingDir { "build/pypkg" } - commandLine PYTHON3, "-m", "build" + doLast { + File setuptools = project(":Debugger-rmi-trace").findPyDep(".") + exec { + workingDir { "build/pypkg" } + commandLine PYTHON3, "-m", "pip" + args "wheel", "-w", "dist/", "--no-index", "--no-deps" + args "-f", setuptools + args "." + } + } } // At the moment, any module with a python package also distributes it. // We can separate this into `distributePythonPackage` later, if necessary. rootProject.assembleDistribution { + dependsOn(buildPyPackage) def p = this.project def zipPath = getZipPath(p) from (p.assemblePyPackage) { exclude '**/*.pyc' exclude '**/*.pyo' exclude '**/__pycache__/**' + exclude 'dist/*.tar.gz' into { zipPath + "/pypkg" } } } + +ext.distributePyDep = { name -> + File dep = findPyDep(name) + def zipPath = getZipPath(project) + rootProject.assembleDistribution { + into ("${zipPath}/pypkg/dist") { + from dep + } + } +} diff --git a/gradle/support/fetchDependencies.gradle b/gradle/support/fetchDependencies.gradle index 34ca2d2be5..3b7d5f33e3 100644 --- a/gradle/support/fetchDependencies.gradle +++ b/gradle/support/fetchDependencies.gradle @@ -164,6 +164,54 @@ ext.deps = [ url: "https://github.com/NationalSecurityAgency/ghidra-data/raw/Ghidra_${RELEASE_VERSION}/FunctionID/vsOlder_x86.fidb", sha256: "98605c6b6b9214a945d844e41c85860d54649a45bca7873ef6991c0e93720787", destination: FID_DIR + ], + [ + name: "protobuf-3.20.3-py2.py3-none-any.whl", + url: "https://files.pythonhosted.org/packages/8d/14/619e24a4c70df2901e1f4dbc50a6291eb63a759172558df326347dce1f0d/protobuf-3.20.3-py2.py3-none-any.whl", + sha256: "a7ca6d488aa8ff7f329d4c545b2dbad8ac31464f1d8b1c87ad1346717731e4db", + destination: file("${DEPS_DIR}/Debugger-rmi-trace/") + ], + [ + name: "psutil-5.9.8.tar.gz", + url: "https://files.pythonhosted.org/packages/90/c7/6dc0a455d111f68ee43f27793971cf03fe29b6ef972042549db29eec39a2/psutil-5.9.8.tar.gz", + sha256: "6be126e3225486dff286a8fb9a06246a5253f4c7c53b475ea5f5ac934e64194c", + destination: file("${DEPS_DIR}/Debugger-rmi-trace/") + ], + [ + name: "setuptools-68.0.0-py3-none-any.whl", + url: "https://files.pythonhosted.org/packages/c7/42/be1c7bbdd83e1bfb160c94b9cafd8e25efc7400346cf7ccdbdb452c467fa/setuptools-68.0.0-py3-none-any.whl", + sha256: "11e52c67415a381d10d6b462ced9cfb97066179f0e871399e006c4ab101fc85f", + destination: file("${DEPS_DIR}/Debugger-rmi-trace/") + ], + [ + name: "wheel-0.37.1-py2.py3-none-any.whl", + url: "https://files.pythonhosted.org/packages/27/d6/003e593296a85fd6ed616ed962795b2f87709c3eee2bca4f6d0fe55c6d00/wheel-0.37.1-py2.py3-none-any.whl", + sha256: "4bdcd7d840138086126cd09254dc6195fb4fc6f01c050a1d7236f2630db1d22a", + destination: file("${DEPS_DIR}/Debugger-rmi-trace/") + ], + [ + name: "Pybag-2.2.10-py3-none-any.whl", + url: "https://files.pythonhosted.org/packages/2d/e0/f877c91e036fcaed2a827f80d6cbdf1d26cffc3333c9ebda31c55c45f050/Pybag-2.2.10-py3-none-any.whl", + sha256: "81cf1e33dd667dd217dc56a123326796e6799b8569f2c6efb78c16375caf9b2b", + destination: file("${DEPS_DIR}/Debugger-agent-dbgeng/") + ], + [ + name: "capstone-5.0.1-py3-none-win_amd64.whl", + url: "https://files.pythonhosted.org/packages/d0/dd/b28df50316ca193dd1275a4c47115a720796d9e1501c1888c4bfa5dc2260/capstone-5.0.1-py3-none-win_amd64.whl", + sha256: "1bfa5c81e6880caf41a31946cd6d2d069c048bcc22edf121254b501a048de675", + destination: file("${DEPS_DIR}/Debugger-agent-dbgeng/") + ], + [ + name: "comtypes-1.4.1-py3-none-any.whl", + url: "https://files.pythonhosted.org/packages/50/8f/518a37381e55a8857a638afa86143efa5508434613541402d20611a1b322/comtypes-1.4.1-py3-none-any.whl", + sha256: "a208a0e3ca1c0a5362735da0ff661822801dce87312b894d7d752add010a21b0", + destination: file("${DEPS_DIR}/Debugger-agent-dbgeng/") + ], + [ + name: "pywin32-306-cp312-cp312-win_amd64.whl", + url: "https://files.pythonhosted.org/packages/83/1c/25b79fc3ec99b19b0a0730cc47356f7e2959863bf9f3cd314332bddb4f68/pywin32-306-cp312-cp312-win_amd64.whl", + sha256: "37257794c1ad39ee9be652da0462dc2e394c8159dfd913a8a4e8eb6fd346da0e", + destination: file("${DEPS_DIR}/Debugger-agent-dbgeng/") ] ] diff --git a/licenses/BSD-3-CAPSTONE.txt b/licenses/BSD-3-CAPSTONE.txt new file mode 100644 index 0000000000..a39d5892d4 --- /dev/null +++ b/licenses/BSD-3-CAPSTONE.txt @@ -0,0 +1,31 @@ +This is the software license for Capstone disassembly framework. +Capstone has been designed & implemented by Nguyen Anh Quynh + +See http://www.capstone-engine.org for further information. + +Copyright (c) 2013, COSEINC. +All rights reserved. + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions are met: + +* Redistributions of source code must retain the above copyright notice, + this list of conditions and the following disclaimer. +* Redistributions in binary form must reproduce the above copyright notice, + this list of conditions and the following disclaimer in the documentation + and/or other materials provided with the distribution. +* Neither the name of the developer(s) nor the names of its + contributors may be used to endorse or promote products derived from this + software without specific prior written permission. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" +AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE +LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR +CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF +SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS +INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN +CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) +ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE +POSSIBILITY OF SUCH DAMAGE. \ No newline at end of file diff --git a/licenses/BSD-3-PSUTIL.txt b/licenses/BSD-3-PSUTIL.txt new file mode 100644 index 0000000000..568faf8f63 --- /dev/null +++ b/licenses/BSD-3-PSUTIL.txt @@ -0,0 +1,29 @@ +BSD 3-Clause License + +Copyright (c) 2009, Jay Loden, Dave Daeschler, Giampaolo Rodola +All rights reserved. + +Redistribution and use in source and binary forms, with or without modification, +are permitted provided that the following conditions are met: + + * Redistributions of source code must retain the above copyright notice, this + list of conditions and the following disclaimer. + + * Redistributions in binary form must reproduce the above copyright notice, + this list of conditions and the following disclaimer in the documentation + and/or other materials provided with the distribution. + + * Neither the name of the psutil authors nor the names of its contributors + may be used to endorse or promote products derived from this software without + specific prior written permission. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND +ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED +WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE +DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR +ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES +(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; +LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON +ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS +SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. \ No newline at end of file diff --git a/licenses/Python_Software_Foundation_License.txt b/licenses/Python_Software_Foundation_License.txt new file mode 100644 index 0000000000..c96142b382 --- /dev/null +++ b/licenses/Python_Software_Foundation_License.txt @@ -0,0 +1,279 @@ +A. HISTORY OF THE SOFTWARE +========================== + +Python was created in the early 1990s by Guido van Rossum at Stichting +Mathematisch Centrum (CWI, see https://www.cwi.nl) in the Netherlands +as a successor of a language called ABC. Guido remains Python's +principal author, although it includes many contributions from others. + +In 1995, Guido continued his work on Python at the Corporation for +National Research Initiatives (CNRI, see https://www.cnri.reston.va.us) +in Reston, Virginia where he released several versions of the +software. + +In May 2000, Guido and the Python core development team moved to +BeOpen.com to form the BeOpen PythonLabs team. In October of the same +year, the PythonLabs team moved to Digital Creations, which became +Zope Corporation. In 2001, the Python Software Foundation (PSF, see +https://www.python.org/psf/) was formed, a non-profit organization +created specifically to own Python-related Intellectual Property. +Zope Corporation was a sponsoring member of the PSF. + +All Python releases are Open Source (see https://opensource.org for +the Open Source Definition). Historically, most, but not all, Python +releases have also been GPL-compatible; the table below summarizes +the various releases. + + Release Derived Year Owner GPL- + from compatible? (1) + + 0.9.0 thru 1.2 1991-1995 CWI yes + 1.3 thru 1.5.2 1.2 1995-1999 CNRI yes + 1.6 1.5.2 2000 CNRI no + 2.0 1.6 2000 BeOpen.com no + 1.6.1 1.6 2001 CNRI yes (2) + 2.1 2.0+1.6.1 2001 PSF no + 2.0.1 2.0+1.6.1 2001 PSF yes + 2.1.1 2.1+2.0.1 2001 PSF yes + 2.1.2 2.1.1 2002 PSF yes + 2.1.3 2.1.2 2002 PSF yes + 2.2 and above 2.1.1 2001-now PSF yes + +Footnotes: + +(1) GPL-compatible doesn't mean that we're distributing Python under + the GPL. All Python licenses, unlike the GPL, let you distribute + a modified version without making your changes open source. The + GPL-compatible licenses make it possible to combine Python with + other software that is released under the GPL; the others don't. + +(2) According to Richard Stallman, 1.6.1 is not GPL-compatible, + because its license has a choice of law clause. According to + CNRI, however, Stallman's lawyer has told CNRI's lawyer that 1.6.1 + is "not incompatible" with the GPL. + +Thanks to the many outside volunteers who have worked under Guido's +direction to make these releases possible. + + +B. TERMS AND CONDITIONS FOR ACCESSING OR OTHERWISE USING PYTHON +=============================================================== + +Python software and documentation are licensed under the +Python Software Foundation License Version 2. + +Starting with Python 3.8.6, examples, recipes, and other code in +the documentation are dual licensed under the PSF License Version 2 +and the Zero-Clause BSD license. + +Some software incorporated into Python is under different licenses. +The licenses are listed with code falling under that license. + + +PYTHON SOFTWARE FOUNDATION LICENSE VERSION 2 +-------------------------------------------- + +1. This LICENSE AGREEMENT is between the Python Software Foundation +("PSF"), and the Individual or Organization ("Licensee") accessing and +otherwise using this software ("Python") in source or binary form and +its associated documentation. + +2. Subject to the terms and conditions of this License Agreement, PSF hereby +grants Licensee a nonexclusive, royalty-free, world-wide license to reproduce, +analyze, test, perform and/or display publicly, prepare derivative works, +distribute, and otherwise use Python alone or in any derivative version, +provided, however, that PSF's License Agreement and PSF's notice of copyright, +i.e., "Copyright (c) 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010, +2011, 2012, 2013, 2014, 2015, 2016, 2017, 2018, 2019, 2020, 2021, 2022, 2023 Python Software Foundation; +All Rights Reserved" are retained in Python alone or in any derivative version +prepared by Licensee. + +3. In the event Licensee prepares a derivative work that is based on +or incorporates Python or any part thereof, and wants to make +the derivative work available to others as provided herein, then +Licensee hereby agrees to include in any such work a brief summary of +the changes made to Python. + +4. PSF is making Python available to Licensee on an "AS IS" +basis. PSF MAKES NO REPRESENTATIONS OR WARRANTIES, EXPRESS OR +IMPLIED. BY WAY OF EXAMPLE, BUT NOT LIMITATION, PSF MAKES NO AND +DISCLAIMS ANY REPRESENTATION OR WARRANTY OF MERCHANTABILITY OR FITNESS +FOR ANY PARTICULAR PURPOSE OR THAT THE USE OF PYTHON WILL NOT +INFRINGE ANY THIRD PARTY RIGHTS. + +5. PSF SHALL NOT BE LIABLE TO LICENSEE OR ANY OTHER USERS OF PYTHON +FOR ANY INCIDENTAL, SPECIAL, OR CONSEQUENTIAL DAMAGES OR LOSS AS +A RESULT OF MODIFYING, DISTRIBUTING, OR OTHERWISE USING PYTHON, +OR ANY DERIVATIVE THEREOF, EVEN IF ADVISED OF THE POSSIBILITY THEREOF. + +6. This License Agreement will automatically terminate upon a material +breach of its terms and conditions. + +7. Nothing in this License Agreement shall be deemed to create any +relationship of agency, partnership, or joint venture between PSF and +Licensee. This License Agreement does not grant permission to use PSF +trademarks or trade name in a trademark sense to endorse or promote +products or services of Licensee, or any third party. + +8. By copying, installing or otherwise using Python, Licensee +agrees to be bound by the terms and conditions of this License +Agreement. + + +BEOPEN.COM LICENSE AGREEMENT FOR PYTHON 2.0 +------------------------------------------- + +BEOPEN PYTHON OPEN SOURCE LICENSE AGREEMENT VERSION 1 + +1. This LICENSE AGREEMENT is between BeOpen.com ("BeOpen"), having an +office at 160 Saratoga Avenue, Santa Clara, CA 95051, and the +Individual or Organization ("Licensee") accessing and otherwise using +this software in source or binary form and its associated +documentation ("the Software"). + +2. Subject to the terms and conditions of this BeOpen Python License +Agreement, BeOpen hereby grants Licensee a non-exclusive, +royalty-free, world-wide license to reproduce, analyze, test, perform +and/or display publicly, prepare derivative works, distribute, and +otherwise use the Software alone or in any derivative version, +provided, however, that the BeOpen Python License is retained in the +Software, alone or in any derivative version prepared by Licensee. + +3. BeOpen is making the Software available to Licensee on an "AS IS" +basis. BEOPEN MAKES NO REPRESENTATIONS OR WARRANTIES, EXPRESS OR +IMPLIED. BY WAY OF EXAMPLE, BUT NOT LIMITATION, BEOPEN MAKES NO AND +DISCLAIMS ANY REPRESENTATION OR WARRANTY OF MERCHANTABILITY OR FITNESS +FOR ANY PARTICULAR PURPOSE OR THAT THE USE OF THE SOFTWARE WILL NOT +INFRINGE ANY THIRD PARTY RIGHTS. + +4. BEOPEN SHALL NOT BE LIABLE TO LICENSEE OR ANY OTHER USERS OF THE +SOFTWARE FOR ANY INCIDENTAL, SPECIAL, OR CONSEQUENTIAL DAMAGES OR LOSS +AS A RESULT OF USING, MODIFYING OR DISTRIBUTING THE SOFTWARE, OR ANY +DERIVATIVE THEREOF, EVEN IF ADVISED OF THE POSSIBILITY THEREOF. + +5. This License Agreement will automatically terminate upon a material +breach of its terms and conditions. + +6. This License Agreement shall be governed by and interpreted in all +respects by the law of the State of California, excluding conflict of +law provisions. Nothing in this License Agreement shall be deemed to +create any relationship of agency, partnership, or joint venture +between BeOpen and Licensee. This License Agreement does not grant +permission to use BeOpen trademarks or trade names in a trademark +sense to endorse or promote products or services of Licensee, or any +third party. As an exception, the "BeOpen Python" logos available at +http://www.pythonlabs.com/logos.html may be used according to the +permissions granted on that web page. + +7. By copying, installing or otherwise using the software, Licensee +agrees to be bound by the terms and conditions of this License +Agreement. + + +CNRI LICENSE AGREEMENT FOR PYTHON 1.6.1 +--------------------------------------- + +1. This LICENSE AGREEMENT is between the Corporation for National +Research Initiatives, having an office at 1895 Preston White Drive, +Reston, VA 20191 ("CNRI"), and the Individual or Organization +("Licensee") accessing and otherwise using Python 1.6.1 software in +source or binary form and its associated documentation. + +2. Subject to the terms and conditions of this License Agreement, CNRI +hereby grants Licensee a nonexclusive, royalty-free, world-wide +license to reproduce, analyze, test, perform and/or display publicly, +prepare derivative works, distribute, and otherwise use Python 1.6.1 +alone or in any derivative version, provided, however, that CNRI's +License Agreement and CNRI's notice of copyright, i.e., "Copyright (c) +1995-2001 Corporation for National Research Initiatives; All Rights +Reserved" are retained in Python 1.6.1 alone or in any derivative +version prepared by Licensee. Alternately, in lieu of CNRI's License +Agreement, Licensee may substitute the following text (omitting the +quotes): "Python 1.6.1 is made available subject to the terms and +conditions in CNRI's License Agreement. This Agreement together with +Python 1.6.1 may be located on the internet using the following +unique, persistent identifier (known as a handle): 1895.22/1013. This +Agreement may also be obtained from a proxy server on the internet +using the following URL: http://hdl.handle.net/1895.22/1013". + +3. In the event Licensee prepares a derivative work that is based on +or incorporates Python 1.6.1 or any part thereof, and wants to make +the derivative work available to others as provided herein, then +Licensee hereby agrees to include in any such work a brief summary of +the changes made to Python 1.6.1. + +4. CNRI is making Python 1.6.1 available to Licensee on an "AS IS" +basis. CNRI MAKES NO REPRESENTATIONS OR WARRANTIES, EXPRESS OR +IMPLIED. BY WAY OF EXAMPLE, BUT NOT LIMITATION, CNRI MAKES NO AND +DISCLAIMS ANY REPRESENTATION OR WARRANTY OF MERCHANTABILITY OR FITNESS +FOR ANY PARTICULAR PURPOSE OR THAT THE USE OF PYTHON 1.6.1 WILL NOT +INFRINGE ANY THIRD PARTY RIGHTS. + +5. CNRI SHALL NOT BE LIABLE TO LICENSEE OR ANY OTHER USERS OF PYTHON +1.6.1 FOR ANY INCIDENTAL, SPECIAL, OR CONSEQUENTIAL DAMAGES OR LOSS AS +A RESULT OF MODIFYING, DISTRIBUTING, OR OTHERWISE USING PYTHON 1.6.1, +OR ANY DERIVATIVE THEREOF, EVEN IF ADVISED OF THE POSSIBILITY THEREOF. + +6. This License Agreement will automatically terminate upon a material +breach of its terms and conditions. + +7. This License Agreement shall be governed by the federal +intellectual property law of the United States, including without +limitation the federal copyright law, and, to the extent such +U.S. federal law does not apply, by the law of the Commonwealth of +Virginia, excluding Virginia's conflict of law provisions. +Notwithstanding the foregoing, with regard to derivative works based +on Python 1.6.1 that incorporate non-separable material that was +previously distributed under the GNU General Public License (GPL), the +law of the Commonwealth of Virginia shall govern this License +Agreement only as to issues arising under or with respect to +Paragraphs 4, 5, and 7 of this License Agreement. Nothing in this +License Agreement shall be deemed to create any relationship of +agency, partnership, or joint venture between CNRI and Licensee. This +License Agreement does not grant permission to use CNRI trademarks or +trade name in a trademark sense to endorse or promote products or +services of Licensee, or any third party. + +8. By clicking on the "ACCEPT" button where indicated, or by copying, +installing or otherwise using Python 1.6.1, Licensee agrees to be +bound by the terms and conditions of this License Agreement. + + ACCEPT + + +CWI LICENSE AGREEMENT FOR PYTHON 0.9.0 THROUGH 1.2 +-------------------------------------------------- + +Copyright (c) 1991 - 1995, Stichting Mathematisch Centrum Amsterdam, +The Netherlands. All rights reserved. + +Permission to use, copy, modify, and distribute this software and its +documentation for any purpose and without fee is hereby granted, +provided that the above copyright notice appear in all copies and that +both that copyright notice and this permission notice appear in +supporting documentation, and that the name of Stichting Mathematisch +Centrum or CWI not be used in advertising or publicity pertaining to +distribution of the software without specific, written prior +permission. + +STICHTING MATHEMATISCH CENTRUM DISCLAIMS ALL WARRANTIES WITH REGARD TO +THIS SOFTWARE, INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND +FITNESS, IN NO EVENT SHALL STICHTING MATHEMATISCH CENTRUM BE LIABLE +FOR ANY SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES +WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN +ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT +OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. + +ZERO-CLAUSE BSD LICENSE FOR CODE IN THE PYTHON DOCUMENTATION +---------------------------------------------------------------------- + +Permission to use, copy, modify, and/or distribute this software for any +purpose with or without fee is hereby granted. + +THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES WITH +REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY +AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY SPECIAL, DIRECT, +INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM +LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR +OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR +PERFORMANCE OF THIS SOFTWARE. \ No newline at end of file diff --git a/licenses/certification.manifest b/licenses/certification.manifest index b3b71658f4..7a4ca7cbb6 100644 --- a/licenses/certification.manifest +++ b/licenses/certification.manifest @@ -3,12 +3,14 @@ Apache_License_2.0.txt||LICENSE||||END| Apache_License_2.0_with_LLVM_Exceptions.txt||LICENSE||||END| BSD-2-ORACLE.txt||LICENSE||||END| BSD-3-APPLE.txt||LICENSE||||END| +BSD-3-CAPSTONE.txt||LICENSE||||END| BSD-3-GOOGLE.txt||LICENSE||||END| BSD-3-GRUVER.txt||LICENSE||||END| BSD-3-HAMCREST.txt||LICENSE||||END| BSD-3-JUNG.txt||LICENSE||||END| BSD-3-ORACLE.txt||LICENSE||||END| BSD-3-PARR-HARWELL.txt||LICENSE||||END| +BSD-3-PSUTIL.txt||LICENSE||||END| BSD.txt||LICENSE||||END| Bouncy_Castle_License.txt||LICENSE||||END| Creative_Commons_Attribution_2.5.html||LICENSE||||END| @@ -30,5 +32,6 @@ Oxygen_Icons_-_LGPL_3.0.txt||LICENSE||||END| PostgresqlJDBC_License.txt||LICENSE||||END| Postgresql_License.txt||LICENSE||||END| Public_Domain.txt||LICENSE||||END| +Python_Software_Foundation_License.txt||LICENSE||||END| Tango_Icons_-_Public_Domain.txt||LICENSE||||END| zlib_License.txt||LICENSE||||END|