Revert "MFV: xz 5.6.0"

This commit reverts 8db56defa7,
rolling back the vendor import of xz 5.6.0 and restoring the
package to version 5.4.5.

The revert was not directly due to the attack (CVE-2024-3094):
our import process have removed the test cases and build scripts
that would have enabled the attack. However, reverting would
help to reduce potential confusion and false positives from
security scanners that assess risk based solely on version
numbers.

Another commit will follow to restore binary compatibility with
the liblzma 5.6.0 library by making the previously private
symbol (lzma_mt_block_size) public.

PR:		278127

(cherry picked from commit 2f9cd13d6c)
This commit is contained in:
Xin LI 2024-04-04 23:39:23 -07:00
parent 9031978083
commit 6b55e41b3f
221 changed files with 10796 additions and 12990 deletions

View file

@ -9,13 +9,15 @@ Authors of XZ Utils
specifically the LZMA SDK <https://7-zip.org/sdk.html>. Without
this code, XZ Utils wouldn't exist.
The SHA-256 implementation in liblzma is based on code written by
Wei Dai in Crypto++ Library <https://www.cryptopp.com/>.
The SHA-256 implementation in liblzma is based on the code found from
7-Zip <https://7-zip.org/>, which has a modified version of the SHA-256
code found from Crypto++ <https://www.cryptopp.com/>. The SHA-256 code
in Crypto++ was written by Kevin Springle and Wei Dai.
A few scripts have been adapted from GNU gzip. The original
versions were written by Jean-loup Gailly, Charles Levert, and
Paul Eggert. Andrew Dudman helped adapting the scripts and their
man pages for XZ Utils.
Some scripts have been adapted from gzip. The original versions
were written by Jean-loup Gailly, Charles Levert, and Paul Eggert.
Andrew Dudman helped adapting the scripts and their man pages for
XZ Utils.
The initial version of the threaded .xz decompressor was written
by Sebastian Andrzej Siewior.
@ -23,31 +25,15 @@ Authors of XZ Utils
The initial version of the .lz (lzip) decoder was written
by Michał Górny.
Architecture-specific CRC optimizations were contributed by
Ilya Kurdyukov, Hans Jansen, and Chenxi Mao.
CLMUL-accelerated CRC code was contributed by Ilya Kurdyukov.
Other authors:
- Jonathan Nieder
- Joachim Henke
Many people have contributed improvements or reported bugs.
Most of these people are mentioned in the file THANKS.
The GNU Autotools-based build system contains files from many authors,
which I'm not trying to list here.
The translations of the command line tools and man pages have been
contributed by many people via the Translation Project:
- https://translationproject.org/domain/xz.html
- https://translationproject.org/domain/xz-man.html
The authors of the translated man pages are in the header comments
of the man page files. In the source package, the authors of the
translations are in po/*.po and po4a/*.po files.
Third-party code whose authors aren't listed here:
- GNU getopt_long() in the 'lib' directory is included for
platforms that don't have a usable getopt_long().
- The build system files from GNU Autoconf, GNU Automake,
GNU Libtool, GNU Gettext, Autoconf Archive, and related files.
Several people have contributed fixes or reported bugs. Most of them
are mentioned in the file THANKS.

View file

@ -6,95 +6,71 @@ XZ Utils Licensing
is a rough summary of which licenses apply to which parts of this
package (but check the individual files to be sure!):
- liblzma is under the BSD Zero Clause License (0BSD).
- liblzma is in the public domain.
- The command line tools xz, xzdec, lzmadec, and lzmainfo are
under 0BSD except that, on systems that don't have a usable
getopt_long, GNU getopt_long is compiled and linked in from the
'lib' directory. The getopt_long code is under GNU LGPLv2.1+.
- xz, xzdec, and lzmadec command line tools are in the public
domain unless GNU getopt_long had to be compiled and linked
in from the lib directory. The getopt_long code is under
GNU LGPLv2.1+.
- The scripts to grep, diff, and view compressed files have been
adapted from GNU gzip. These scripts (xzgrep, xzdiff, xzless,
and xzmore) are under GNU GPLv2+. The man pages of the scripts
are under 0BSD; they aren't based on the man pages of GNU gzip.
adapted from gzip. These scripts and their documentation are
under GNU GPLv2+.
- Most of the XZ Utils specific documentation that is in
plain text files (like README, INSTALL, PACKAGERS, NEWS,
and ChangeLog) are under 0BSD unless stated otherwise in
the file itself. The files xz-file-format.txt and
lzma-file-format.xt are in the public domain but may
be distributed under the terms of 0BSD too.
- Doxygen-generated HTML version of the liblzma API documentation:
While Doxygen is under the GNU GPLv2, the license information
in Doxygen includes the following exception:
Documents produced by doxygen are derivative works
derived from the input used in their production;
they are not affected by this license.
- All the documentation in the doc directory and most of the
XZ Utils specific documentation files in other directories
are in the public domain.
Note: The JavaScript files (under the MIT license) have
been removed from the Doxygen output.
been removed from the Doxygen-generated HTML version of the
liblzma API documentation. Doxygen itself is under the GNU GPL
but the remaining files generated by Doxygen are not affected
by the licenses used in Doxygen because Doxygen licensing has
the following exception:
- The XZ logo (xz-logo.png) included in the Doxygen-generated
documentation is under the Creative Commons BY-SA 4.0 license.
"Documents produced by doxygen are derivative works
derived from the input used in their production;
they are not affected by this license."
- Translated messages and man pages are under 0BSD except that
some old translations are in the public domain.
- Translated messages are in the public domain.
- Test files and test code in the 'tests' directory, and
debugging utilities in the 'debug' directory are under
the BSD Zero Clause License (0BSD).
- The build system contains public domain files, and files that
are under GNU GPLv2+ or GNU GPLv3+. None of these files end up
in the binaries being built.
- The GNU Autotools based build system contains files that are
under GNU GPLv2+, GNU GPLv3+, and a few permissive licenses.
These files don't affect the licensing of the binaries being
built.
- Test files and test code in the tests directory, and debugging
utilities in the debug directory are in the public domain.
- The extra directory contain files that are under various
free software licenses.
- The extra directory may contain public domain files, and files
that are under various free software licenses.
For the files under the BSD Zero Clause License (0BSD), if
a copyright notice is needed, the following is sufficient:
You can do whatever you want with the files that have been put into
the public domain. If you find public domain legally problematic,
take the previous sentence as a license grant. If you still find
the lack of copyright legally problematic, you have too many
lawyers.
Copyright (C) The XZ Utils authors and contributors
As usual, this software is provided "as is", without any warranty.
If you copy significant amounts of 0BSD-licensed code from XZ Utils
If you copy significant amounts of public domain code from XZ Utils
into your project, acknowledging this somewhere in your software is
polite (especially if it is proprietary, non-free software), but
it is not legally required by the license terms. Here is an example
of a good notice to put into "about box" or into documentation:
naturally it is not legally required. Here is an example of a good
notice to put into "about box" or into documentation:
This software includes code from XZ Utils
<https://xz.tukaani.org/xz-utils/>.
This software includes code from XZ Utils <https://tukaani.org/xz/>.
The following license texts are included in the following files:
- COPYING.0BSD: BSD Zero Clause License
- COPYING.LGPLv2.1: GNU Lesser General Public License version 2.1
- COPYING.GPLv2: GNU General Public License version 2
- COPYING.GPLv3: GNU General Public License version 3
- COPYING.CC-BY-SA-4.0: Creative Commons Attribution-ShareAlike 4.0
International Public License
A note about old XZ Utils releases:
Note that the toolchain (compiler, linker etc.) may add some code
pieces that are copyrighted. Thus, it is possible that e.g. liblzma
binary wouldn't actually be in the public domain in its entirety
even though it contains no copyrighted code from the XZ Utils source
package.
XZ Utils releases 5.4.6 and older and 5.5.1alpha have a
significant amount of code put into the public domain and
that obviously remains so. The switch from public domain to
0BSD for newer releases was made in Febrary 2024 because
public domain has (real or perceived) legal ambiguities in
some jurisdictions.
There is very little *practical* difference between public
domain and 0BSD. The main difference likely is that one
shouldn't claim that 0BSD-licensed code is in the public
domain; 0BSD-licensed code is copyrighted but available under
an extremely permissive license. Neither 0BSD nor public domain
require retaining or reproducing author, copyright holder, or
license notices when distributing the software. (Compare to,
for example, BSD 2-Clause "Simplified" License which does have
such requirements.)
If you have questions, don't hesitate to ask for more information.
The contact information is in the README file.
If you have questions, don't hesitate to ask the author(s) for more
information.

View file

@ -1,11 +0,0 @@
Permission to use, copy, modify, and/or distribute this
software for any purpose with or without fee is hereby granted.
THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL
WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL
THE AUTHOR BE LIABLE FOR ANY SPECIAL, DIRECT, INDIRECT, OR
CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM
LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT,
NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN
CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.

View file

@ -1,427 +0,0 @@
Attribution-ShareAlike 4.0 International
=======================================================================
Creative Commons Corporation ("Creative Commons") is not a law firm and
does not provide legal services or legal advice. Distribution of
Creative Commons public licenses does not create a lawyer-client or
other relationship. Creative Commons makes its licenses and related
information available on an "as-is" basis. Creative Commons gives no
warranties regarding its licenses, any material licensed under their
terms and conditions, or any related information. Creative Commons
disclaims all liability for damages resulting from their use to the
fullest extent possible.
Using Creative Commons Public Licenses
Creative Commons public licenses provide a standard set of terms and
conditions that creators and other rights holders may use to share
original works of authorship and other material subject to copyright
and certain other rights specified in the public license below. The
following considerations are for informational purposes only, are not
exhaustive, and do not form part of our licenses.
Considerations for licensors: Our public licenses are
intended for use by those authorized to give the public
permission to use material in ways otherwise restricted by
copyright and certain other rights. Our licenses are
irrevocable. Licensors should read and understand the terms
and conditions of the license they choose before applying it.
Licensors should also secure all rights necessary before
applying our licenses so that the public can reuse the
material as expected. Licensors should clearly mark any
material not subject to the license. This includes other CC-
licensed material, or material used under an exception or
limitation to copyright. More considerations for licensors:
wiki.creativecommons.org/Considerations_for_licensors
Considerations for the public: By using one of our public
licenses, a licensor grants the public permission to use the
licensed material under specified terms and conditions. If
the licensor's permission is not necessary for any reason--for
example, because of any applicable exception or limitation to
copyright--then that use is not regulated by the license. Our
licenses grant only permissions under copyright and certain
other rights that a licensor has authority to grant. Use of
the licensed material may still be restricted for other
reasons, including because others have copyright or other
rights in the material. A licensor may make special requests,
such as asking that all changes be marked or described.
Although not required by our licenses, you are encouraged to
respect those requests where reasonable. More considerations
for the public:
wiki.creativecommons.org/Considerations_for_licensees
=======================================================================
Creative Commons Attribution-ShareAlike 4.0 International Public
License
By exercising the Licensed Rights (defined below), You accept and agree
to be bound by the terms and conditions of this Creative Commons
Attribution-ShareAlike 4.0 International Public License ("Public
License"). To the extent this Public License may be interpreted as a
contract, You are granted the Licensed Rights in consideration of Your
acceptance of these terms and conditions, and the Licensor grants You
such rights in consideration of benefits the Licensor receives from
making the Licensed Material available under these terms and
conditions.
Section 1 -- Definitions.
a. Adapted Material means material subject to Copyright and Similar
Rights that is derived from or based upon the Licensed Material
and in which the Licensed Material is translated, altered,
arranged, transformed, or otherwise modified in a manner requiring
permission under the Copyright and Similar Rights held by the
Licensor. For purposes of this Public License, where the Licensed
Material is a musical work, performance, or sound recording,
Adapted Material is always produced where the Licensed Material is
synched in timed relation with a moving image.
b. Adapter's License means the license You apply to Your Copyright
and Similar Rights in Your contributions to Adapted Material in
accordance with the terms and conditions of this Public License.
c. BY-SA Compatible License means a license listed at
creativecommons.org/compatiblelicenses, approved by Creative
Commons as essentially the equivalent of this Public License.
d. Copyright and Similar Rights means copyright and/or similar rights
closely related to copyright including, without limitation,
performance, broadcast, sound recording, and Sui Generis Database
Rights, without regard to how the rights are labeled or
categorized. For purposes of this Public License, the rights
specified in Section 2(b)(1)-(2) are not Copyright and Similar
Rights.
e. Effective Technological Measures means those measures that, in the
absence of proper authority, may not be circumvented under laws
fulfilling obligations under Article 11 of the WIPO Copyright
Treaty adopted on December 20, 1996, and/or similar international
agreements.
f. Exceptions and Limitations means fair use, fair dealing, and/or
any other exception or limitation to Copyright and Similar Rights
that applies to Your use of the Licensed Material.
g. License Elements means the license attributes listed in the name
of a Creative Commons Public License. The License Elements of this
Public License are Attribution and ShareAlike.
h. Licensed Material means the artistic or literary work, database,
or other material to which the Licensor applied this Public
License.
i. Licensed Rights means the rights granted to You subject to the
terms and conditions of this Public License, which are limited to
all Copyright and Similar Rights that apply to Your use of the
Licensed Material and that the Licensor has authority to license.
j. Licensor means the individual(s) or entity(ies) granting rights
under this Public License.
k. Share means to provide material to the public by any means or
process that requires permission under the Licensed Rights, such
as reproduction, public display, public performance, distribution,
dissemination, communication, or importation, and to make material
available to the public including in ways that members of the
public may access the material from a place and at a time
individually chosen by them.
l. Sui Generis Database Rights means rights other than copyright
resulting from Directive 96/9/EC of the European Parliament and of
the Council of 11 March 1996 on the legal protection of databases,
as amended and/or succeeded, as well as other essentially
equivalent rights anywhere in the world.
m. You means the individual or entity exercising the Licensed Rights
under this Public License. Your has a corresponding meaning.
Section 2 -- Scope.
a. License grant.
1. Subject to the terms and conditions of this Public License,
the Licensor hereby grants You a worldwide, royalty-free,
non-sublicensable, non-exclusive, irrevocable license to
exercise the Licensed Rights in the Licensed Material to:
a. reproduce and Share the Licensed Material, in whole or
in part; and
b. produce, reproduce, and Share Adapted Material.
2. Exceptions and Limitations. For the avoidance of doubt, where
Exceptions and Limitations apply to Your use, this Public
License does not apply, and You do not need to comply with
its terms and conditions.
3. Term. The term of this Public License is specified in Section
6(a).
4. Media and formats; technical modifications allowed. The
Licensor authorizes You to exercise the Licensed Rights in
all media and formats whether now known or hereafter created,
and to make technical modifications necessary to do so. The
Licensor waives and/or agrees not to assert any right or
authority to forbid You from making technical modifications
necessary to exercise the Licensed Rights, including
technical modifications necessary to circumvent Effective
Technological Measures. For purposes of this Public License,
simply making modifications authorized by this Section 2(a)
(4) never produces Adapted Material.
5. Downstream recipients.
a. Offer from the Licensor -- Licensed Material. Every
recipient of the Licensed Material automatically
receives an offer from the Licensor to exercise the
Licensed Rights under the terms and conditions of this
Public License.
b. Additional offer from the Licensor -- Adapted Material.
Every recipient of Adapted Material from You
automatically receives an offer from the Licensor to
exercise the Licensed Rights in the Adapted Material
under the conditions of the Adapter's License You apply.
c. No downstream restrictions. You may not offer or impose
any additional or different terms or conditions on, or
apply any Effective Technological Measures to, the
Licensed Material if doing so restricts exercise of the
Licensed Rights by any recipient of the Licensed
Material.
6. No endorsement. Nothing in this Public License constitutes or
may be construed as permission to assert or imply that You
are, or that Your use of the Licensed Material is, connected
with, or sponsored, endorsed, or granted official status by,
the Licensor or others designated to receive attribution as
provided in Section 3(a)(1)(A)(i).
b. Other rights.
1. Moral rights, such as the right of integrity, are not
licensed under this Public License, nor are publicity,
privacy, and/or other similar personality rights; however, to
the extent possible, the Licensor waives and/or agrees not to
assert any such rights held by the Licensor to the limited
extent necessary to allow You to exercise the Licensed
Rights, but not otherwise.
2. Patent and trademark rights are not licensed under this
Public License.
3. To the extent possible, the Licensor waives any right to
collect royalties from You for the exercise of the Licensed
Rights, whether directly or through a collecting society
under any voluntary or waivable statutory or compulsory
licensing scheme. In all other cases the Licensor expressly
reserves any right to collect such royalties.
Section 3 -- License Conditions.
Your exercise of the Licensed Rights is expressly made subject to the
following conditions.
a. Attribution.
1. If You Share the Licensed Material (including in modified
form), You must:
a. retain the following if it is supplied by the Licensor
with the Licensed Material:
i. identification of the creator(s) of the Licensed
Material and any others designated to receive
attribution, in any reasonable manner requested by
the Licensor (including by pseudonym if
designated);
ii. a copyright notice;
iii. a notice that refers to this Public License;
iv. a notice that refers to the disclaimer of
warranties;
v. a URI or hyperlink to the Licensed Material to the
extent reasonably practicable;
b. indicate if You modified the Licensed Material and
retain an indication of any previous modifications; and
c. indicate the Licensed Material is licensed under this
Public License, and include the text of, or the URI or
hyperlink to, this Public License.
2. You may satisfy the conditions in Section 3(a)(1) in any
reasonable manner based on the medium, means, and context in
which You Share the Licensed Material. For example, it may be
reasonable to satisfy the conditions by providing a URI or
hyperlink to a resource that includes the required
information.
3. If requested by the Licensor, You must remove any of the
information required by Section 3(a)(1)(A) to the extent
reasonably practicable.
b. ShareAlike.
In addition to the conditions in Section 3(a), if You Share
Adapted Material You produce, the following conditions also apply.
1. The Adapter's License You apply must be a Creative Commons
license with the same License Elements, this version or
later, or a BY-SA Compatible License.
2. You must include the text of, or the URI or hyperlink to, the
Adapter's License You apply. You may satisfy this condition
in any reasonable manner based on the medium, means, and
context in which You Share Adapted Material.
3. You may not offer or impose any additional or different terms
or conditions on, or apply any Effective Technological
Measures to, Adapted Material that restrict exercise of the
rights granted under the Adapter's License You apply.
Section 4 -- Sui Generis Database Rights.
Where the Licensed Rights include Sui Generis Database Rights that
apply to Your use of the Licensed Material:
a. for the avoidance of doubt, Section 2(a)(1) grants You the right
to extract, reuse, reproduce, and Share all or a substantial
portion of the contents of the database;
b. if You include all or a substantial portion of the database
contents in a database in which You have Sui Generis Database
Rights, then the database in which You have Sui Generis Database
Rights (but not its individual contents) is Adapted Material,
including for purposes of Section 3(b); and
c. You must comply with the conditions in Section 3(a) if You Share
all or a substantial portion of the contents of the database.
For the avoidance of doubt, this Section 4 supplements and does not
replace Your obligations under this Public License where the Licensed
Rights include other Copyright and Similar Rights.
Section 5 -- Disclaimer of Warranties and Limitation of Liability.
a. UNLESS OTHERWISE SEPARATELY UNDERTAKEN BY THE LICENSOR, TO THE
EXTENT POSSIBLE, THE LICENSOR OFFERS THE LICENSED MATERIAL AS-IS
AND AS-AVAILABLE, AND MAKES NO REPRESENTATIONS OR WARRANTIES OF
ANY KIND CONCERNING THE LICENSED MATERIAL, WHETHER EXPRESS,
IMPLIED, STATUTORY, OR OTHER. THIS INCLUDES, WITHOUT LIMITATION,
WARRANTIES OF TITLE, MERCHANTABILITY, FITNESS FOR A PARTICULAR
PURPOSE, NON-INFRINGEMENT, ABSENCE OF LATENT OR OTHER DEFECTS,
ACCURACY, OR THE PRESENCE OR ABSENCE OF ERRORS, WHETHER OR NOT
KNOWN OR DISCOVERABLE. WHERE DISCLAIMERS OF WARRANTIES ARE NOT
ALLOWED IN FULL OR IN PART, THIS DISCLAIMER MAY NOT APPLY TO YOU.
b. TO THE EXTENT POSSIBLE, IN NO EVENT WILL THE LICENSOR BE LIABLE
TO YOU ON ANY LEGAL THEORY (INCLUDING, WITHOUT LIMITATION,
NEGLIGENCE) OR OTHERWISE FOR ANY DIRECT, SPECIAL, INDIRECT,
INCIDENTAL, CONSEQUENTIAL, PUNITIVE, EXEMPLARY, OR OTHER LOSSES,
COSTS, EXPENSES, OR DAMAGES ARISING OUT OF THIS PUBLIC LICENSE OR
USE OF THE LICENSED MATERIAL, EVEN IF THE LICENSOR HAS BEEN
ADVISED OF THE POSSIBILITY OF SUCH LOSSES, COSTS, EXPENSES, OR
DAMAGES. WHERE A LIMITATION OF LIABILITY IS NOT ALLOWED IN FULL OR
IN PART, THIS LIMITATION MAY NOT APPLY TO YOU.
c. The disclaimer of warranties and limitation of liability provided
above shall be interpreted in a manner that, to the extent
possible, most closely approximates an absolute disclaimer and
waiver of all liability.
Section 6 -- Term and Termination.
a. This Public License applies for the term of the Copyright and
Similar Rights licensed here. However, if You fail to comply with
this Public License, then Your rights under this Public License
terminate automatically.
b. Where Your right to use the Licensed Material has terminated under
Section 6(a), it reinstates:
1. automatically as of the date the violation is cured, provided
it is cured within 30 days of Your discovery of the
violation; or
2. upon express reinstatement by the Licensor.
For the avoidance of doubt, this Section 6(b) does not affect any
right the Licensor may have to seek remedies for Your violations
of this Public License.
c. For the avoidance of doubt, the Licensor may also offer the
Licensed Material under separate terms or conditions or stop
distributing the Licensed Material at any time; however, doing so
will not terminate this Public License.
d. Sections 1, 5, 6, 7, and 8 survive termination of this Public
License.
Section 7 -- Other Terms and Conditions.
a. The Licensor shall not be bound by any additional or different
terms or conditions communicated by You unless expressly agreed.
b. Any arrangements, understandings, or agreements regarding the
Licensed Material not stated herein are separate from and
independent of the terms and conditions of this Public License.
Section 8 -- Interpretation.
a. For the avoidance of doubt, this Public License does not, and
shall not be interpreted to, reduce, limit, restrict, or impose
conditions on any use of the Licensed Material that could lawfully
be made without permission under this Public License.
b. To the extent possible, if any provision of this Public License is
deemed unenforceable, it shall be automatically reformed to the
minimum extent necessary to make it enforceable. If the provision
cannot be reformed, it shall be severed from this Public License
without affecting the enforceability of the remaining terms and
conditions.
c. No term or condition of this Public License will be waived and no
failure to comply consented to unless expressly agreed to by the
Licensor.
d. Nothing in this Public License constitutes or may be interpreted
as a limitation upon, or waiver of, any privileges and immunities
that apply to the Licensor or You, including from the legal
processes of any jurisdiction or authority.
=======================================================================
Creative Commons is not a party to its public
licenses. Notwithstanding, Creative Commons may elect to apply one of
its public licenses to material it publishes and in those instances
will be considered the “Licensor.” The text of the Creative Commons
public licenses is dedicated to the public domain under the CC0 Public
Domain Dedication. Except for the limited purpose of indicating that
material is shared under a Creative Commons public license or as
otherwise permitted by the Creative Commons policies published at
creativecommons.org/policies, Creative Commons does not authorize the
use of the trademark "Creative Commons" or any other trademark or logo
of Creative Commons without its prior written consent including,
without limitation, in connection with any unauthorized modifications
to any of its public licenses or any other arrangements,
understandings, or agreements concerning use of licensed material. For
the avoidance of doubt, this paragraph does not form part of the
public licenses.
Creative Commons may be contacted at creativecommons.org.

File diff suppressed because it is too large Load diff

View file

@ -67,27 +67,24 @@ XZ Utils
1.1. Overall documentation
README This file
README This file
INSTALL.generic Generic install instructions for those not
familiar with packages using GNU Autotools
INSTALL Installation instructions specific to XZ Utils
PACKAGERS Information to packagers of XZ Utils
INSTALL.generic Generic install instructions for those not familiar
with packages using GNU Autotools
INSTALL Installation instructions specific to XZ Utils
PACKAGERS Information to packagers of XZ Utils
COPYING XZ Utils copyright and license information
COPYING.0BSD BSD Zero Clause License
COPYING.GPLv2 GNU General Public License version 2
COPYING.GPLv3 GNU General Public License version 3
COPYING.LGPLv2.1 GNU Lesser General Public License version 2.1
COPYING.CC-BY-SA-4.0 Creative Commons Attribution-ShareAlike 4.0
International Public License
COPYING XZ Utils copyright and license information
COPYING.GPLv2 GNU General Public License version 2
COPYING.GPLv3 GNU General Public License version 3
COPYING.LGPLv2.1 GNU Lesser General Public License version 2.1
AUTHORS The main authors of XZ Utils
THANKS Incomplete list of people who have helped making
this software
NEWS User-visible changes between XZ Utils releases
ChangeLog Detailed list of changes (commit log)
TODO Known bugs and some sort of to-do list
AUTHORS The main authors of XZ Utils
THANKS Incomplete list of people who have helped making
this software
NEWS User-visible changes between XZ Utils releases
ChangeLog Detailed list of changes (commit log)
TODO Known bugs and some sort of to-do list
Note that only some of the above files are included in binary
packages.
@ -290,11 +287,11 @@ XZ Utils
XZ Embedded is a limited implementation written for use in the Linux
kernel, but it is also suitable for other embedded use.
https://xz.tukaani.org/xz-embedded/
https://tukaani.org/xz/embedded.html
XZ for Java is a complete implementation written in pure Java.
https://xz.tukaani.org/xz-for-java/
https://tukaani.org/xz/java.html
6. Contact information

View file

@ -5,7 +5,6 @@ Thanks
Some people have helped more, some less, but nevertheless everyone's help
has been important. :-) In alphabetical order:
- Mark Adler
- Kian-Meng Ang
- H. Peter Anvin
- Jeff Bastian
- Nelson H. F. Beebe
@ -52,11 +51,9 @@ has been important. :-) In alphabetical order:
- Bjarni Ingi Gislason
- John Paul Adrian Glaubitz
- Bill Glessner
- Matthew Good
- Michał Górny
- Jason Gorski
- Juan Manuel Guerrero
- Gabriela Gutierrez
- Diederik de Haas
- Joachim Henke
- Christian Hesse
@ -65,7 +62,6 @@ has been important. :-) In alphabetical order:
- Nicholas Jackson
- Sam James
- Hajin Jang
- Hans Jansen
- Jouk Jansen
- Jun I Jin
- Kiyoshi Kanazawa
@ -83,7 +79,6 @@ has been important. :-) In alphabetical order:
- Ilya Kurdyukov
- Peter Lawler
- James M Leddy
- Kelvin Lee
- Vincent Lefevre
- Hin-Tak Leung
- Andraž 'ruskie' Levstik
@ -94,7 +89,6 @@ has been important. :-) In alphabetical order:
- Lorenzo De Liso
- H.J. Lu
- Bela Lubkin
- Chenxi Mao
- Gregory Margo
- Julien Marrec
- Ed Maste
@ -156,7 +150,6 @@ has been important. :-) In alphabetical order:
- Mohammed Adnène Trojette
- Alexey Tourbin
- Taiki Tsunekawa
- Maksym Vatsyk
- Loganaden Velvindron
- Patrick J. Volkerding
- Martin Väth

View file

@ -24,6 +24,10 @@ Known bugs
tuklib_exit() doesn't block signals => EINTR is possible.
SIGTSTP is not handled. If xz is stopped, the estimated remaining
time and calculated (de)compression speed won't make sense in the
progress indicator (xz --verbose).
If liblzma has created threads and fork() gets called, liblzma
code will break in the child process unless it calls exec() and
doesn't touch liblzma.

View file

@ -61,7 +61,7 @@ PROJECT_BRIEF =
# pixels and the maximum width should not exceed 200 pixels. Doxygen will copy
# the logo to the output directory.
PROJECT_LOGO = ../doc/xz-logo.png
PROJECT_LOGO =
# The OUTPUT_DIRECTORY tag is used to specify the (relative or absolute) path
# into which the generated documentation will be written. If a relative path is
@ -1246,7 +1246,7 @@ HTML_HEADER =
# that doxygen normally uses.
# This tag requires that the tag GENERATE_HTML is set to YES.
HTML_FOOTER = footer.html
HTML_FOOTER =
# The HTML_STYLESHEET tag can be used to specify a user-defined cascading style
# sheet that is used by each HTML page. It can be used to fine-tune the look of
@ -1286,7 +1286,7 @@ HTML_EXTRA_STYLESHEET =
# files will be copied as-is; there are no commands or markers available.
# This tag requires that the tag GENERATE_HTML is set to YES.
HTML_EXTRA_FILES = ../COPYING.CC-BY-SA-4.0
HTML_EXTRA_FILES =
# The HTML_COLORSTYLE tag can be used to specify if the generated HTML output
# should be rendered with a dark or light theme.

View file

@ -1,13 +0,0 @@
<hr class="footer"/>
<p style="text-align: right;padding-right: 12px;">
XZ logo &copy; 2023 by Jia Tan is licensed under
<a href="COPYING.CC-BY-SA-4.0"
rel="license"
style="display:inline-block;">
CC BY-SA 4.0
</a>
</p>
</body>
</html>

View file

@ -1,6 +1,5 @@
#!/bin/sh
# SPDX-License-Identifier: 0BSD
#
#############################################################################
#
# Updates the Doxygen generated documentation files in the source tree.
@ -17,6 +16,9 @@
# Authors: Jia Tan
# Lasse Collin
#
# This file has been put into the public domain.
# You can do whatever you want with this file.
#
#############################################################################
set -e

View file

@ -1,5 +1,3 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file mythread.h
@ -7,6 +5,9 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#ifndef MYTHREAD_H
@ -111,25 +112,6 @@ mythread_sigmask(int how, const sigset_t *restrict set,
# include <sys/time.h>
#endif
// MinGW-w64 with winpthreads:
//
// NOTE: Typical builds with MinGW-w64 don't use this code (MYTHREAD_POSIX).
// Instead, native Windows threading APIs are used (MYTHREAD_VISTA or
// MYTHREAD_WIN95).
//
// MinGW-w64 has _sigset_t (an integer type) in <sys/types.h>.
// If _POSIX was #defined, the header would add the alias sigset_t too.
// Let's keep this working even without _POSIX.
//
// There are no functions that actually do something with sigset_t
// because signals barely exist on Windows. The sigfillset macro below
// is just to silence warnings. There is no sigfillset() in MinGW-w64.
#ifdef __MINGW32__
# include <sys/types.h>
# define sigset_t _sigset_t
# define sigfillset(set_ptr) do { *(set_ptr) = 0; } while (0)
#endif
#define MYTHREAD_RET_TYPE void *
#define MYTHREAD_RET_VALUE NULL
@ -158,13 +140,11 @@ typedef struct timespec mythread_condtime;
// Use pthread_sigmask() to set the signal mask in multi-threaded programs.
// Do nothing on OpenVMS since it lacks pthread_sigmask().
// Do nothing on MinGW-w64 too to silence warnings (its pthread_sigmask()
// is #defined to 0 so it's a no-op).
static inline void
mythread_sigmask(int how, const sigset_t *restrict set,
sigset_t *restrict oset)
{
#if defined(__VMS) || defined(__MINGW32__)
#ifdef __VMS
(void)how;
(void)set;
(void)oset;

View file

@ -1,5 +1,3 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file sysdefs.h
@ -10,6 +8,9 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#ifndef LZMA_SYSDEFS_H
@ -158,16 +159,13 @@ typedef unsigned char _Bool;
#include <string.h>
// Visual Studio 2013 update 2 supports only __inline, not inline.
// MSVC v19.0 / VS 2015 and newer support both.
//
// MSVC v19.27 (VS 2019 version 16.7) added support for restrict.
// Older ones support only __restrict.
#ifdef _MSC_VER
# if _MSC_VER < 1900 && !defined(inline)
// As of MSVC 2013, inline and restrict are supported with
// non-standard keywords.
#if defined(_WIN32) && defined(_MSC_VER)
# ifndef inline
# define inline __inline
# endif
# if _MSC_VER < 1927 && !defined(restrict)
# ifndef restrict
# define restrict __restrict
# endif
#endif

View file

@ -1,5 +1,3 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file tuklib_common.h
@ -7,6 +5,9 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#ifndef TUKLIB_COMMON_H
@ -56,28 +57,8 @@
# define TUKLIB_GNUC_REQ(major, minor) 0
#endif
// tuklib_attr_noreturn attribute is used to mark functions as non-returning.
// We cannot use "noreturn" as the macro name because then C23 code that
// uses [[noreturn]] would break as it would expand to [[ [[noreturn]] ]].
//
// tuklib_attr_noreturn must be used at the beginning of function declaration
// to work in all cases. The [[noreturn]] syntax is the most limiting, it
// must be even before any GNU C's __attribute__ keywords:
//
// tuklib_attr_noreturn
// __attribute__((nonnull(1)))
// extern void foo(const char *s);
//
// FIXME: Update __STDC_VERSION__ for the final C23 version. 202000 is used
// by GCC 13 and Clang 15 with -std=c2x.
#if defined(__STDC_VERSION__) && __STDC_VERSION__ >= 202000
# define tuklib_attr_noreturn [[noreturn]]
#elif defined(__STDC_VERSION__) && __STDC_VERSION__ >= 201112
# define tuklib_attr_noreturn _Noreturn
#elif TUKLIB_GNUC_REQ(2, 5)
#if TUKLIB_GNUC_REQ(2, 5)
# define tuklib_attr_noreturn __attribute__((__noreturn__))
#elif defined(_MSC_VER)
# define tuklib_attr_noreturn __declspec(noreturn)
#else
# define tuklib_attr_noreturn
#endif

View file

@ -1,5 +1,3 @@
// SPDX-License-Identifier: 0BSD
// If config.h isn't available, assume that the headers required by
// tuklib_common.h are available. This is required by crc32_tablegen.c.
#ifdef HAVE_CONFIG_H

View file

@ -1,5 +1,3 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file tuklib_cpucores.c
@ -7,6 +5,9 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#include "tuklib_cpucores.h"

View file

@ -1,5 +1,3 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file tuklib_cpucores.h
@ -7,6 +5,9 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#ifndef TUKLIB_CPUCORES_H

View file

@ -1,5 +1,3 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file tuklib_exit.c
@ -7,6 +5,9 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#include "tuklib_common.h"

View file

@ -1,5 +1,3 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file tuklib_exit.h
@ -8,6 +6,9 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#ifndef TUKLIB_EXIT_H
@ -17,8 +18,8 @@
TUKLIB_DECLS_BEGIN
#define tuklib_exit TUKLIB_SYMBOL(tuklib_exit)
tuklib_attr_noreturn
extern void tuklib_exit(int status, int err_status, int show_error);
extern void tuklib_exit(int status, int err_status, int show_error)
tuklib_attr_noreturn;
TUKLIB_DECLS_END
#endif

View file

@ -1,5 +1,3 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file tuklib_gettext.h
@ -7,6 +5,9 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#ifndef TUKLIB_GETTEXT_H

View file

@ -1,5 +1,3 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file tuklib_integer.h
@ -39,6 +37,9 @@
// Authors: Lasse Collin
// Joachim Henke
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#ifndef TUKLIB_INTEGER_H
@ -250,7 +251,7 @@
// was one instruction longer.
//
// Conclusion: At least in case of GCC and Clang, byte-by-byte code is
// the best choice for strict-align archs to do unaligned access.
// the best choise for strict-align archs to do unaligned access.
//
// See also: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=111502
//

View file

@ -1,5 +1,3 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file tuklib_mbstr.h
@ -12,6 +10,9 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#ifndef TUKLIB_MBSTR_H

View file

@ -1,5 +1,3 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file tuklib_mbstr_fw.c
@ -7,6 +5,9 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#include "tuklib_mbstr.h"

View file

@ -1,5 +1,3 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file tuklib_mbstr_width.c
@ -7,6 +5,9 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#include "tuklib_mbstr.h"

View file

@ -1,5 +1,3 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file tuklib_open_stdxxx.c
@ -7,6 +5,9 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#include "tuklib_open_stdxxx.h"

View file

@ -1,5 +1,3 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file tuklib_open_stdxxx.h
@ -7,6 +5,9 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#ifndef TUKLIB_OPEN_STDXXX_H

View file

@ -1,5 +1,3 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file tuklib_physmem.c
@ -7,6 +5,9 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#include "tuklib_physmem.h"
@ -72,20 +73,23 @@
#endif
// With GCC >= 8.1 with -Wextra and Clang >= 13 with -Wcast-function-type
// will warn about the Windows-specific code.
#if defined(__has_warning)
# if __has_warning("-Wcast-function-type")
# define CAN_DISABLE_WCAST_FUNCTION_TYPE 1
# endif
#elif TUKLIB_GNUC_REQ(8,1)
# define CAN_DISABLE_WCAST_FUNCTION_TYPE 1
#endif
extern uint64_t
tuklib_physmem(void)
{
uint64_t ret = 0;
#if defined(_WIN32) || defined(__CYGWIN__)
// This requires Windows 2000 or later.
MEMORYSTATUSEX meminfo;
meminfo.dwLength = sizeof(meminfo);
if (GlobalMemoryStatusEx(&meminfo))
ret = meminfo.ullTotalPhys;
/*
// Old version that is compatible with even Win95:
if ((GetVersion() & 0xFF) >= 5) {
// Windows 2000 and later have GlobalMemoryStatusEx() which
// supports reporting values greater than 4 GiB. To keep the
@ -121,7 +125,6 @@ tuklib_physmem(void)
GlobalMemoryStatus(&meminfo);
ret = meminfo.dwTotalPhys;
}
*/
#elif defined(__OS2__)
unsigned long mem;

View file

@ -1,5 +1,3 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file tuklib_physmem.h
@ -7,6 +5,9 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#ifndef TUKLIB_PHYSMEM_H

View file

@ -1,5 +1,3 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file tuklib_progname.c
@ -7,6 +5,9 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#include "tuklib_progname.h"

View file

@ -1,5 +1,3 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file tuklib_progname.h
@ -7,6 +5,9 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#ifndef TUKLIB_PROGNAME_H

View file

@ -1,30 +1,31 @@
/* SPDX-License-Identifier: 0BSD */
/**
* \file api/lzma.h
* \brief The public API of liblzma data compression library
* \mainpage
*
* liblzma is a general-purpose data compression library with a zlib-like API.
* The native file format is .xz, but also the old .lzma format and raw (no
* headers) streams are supported. Multiple compression algorithms (filters)
* are supported. Currently LZMA2 is the primary filter.
* liblzma is a public domain general-purpose data compression library with
* a zlib-like API. The native file format is .xz, but also the old .lzma
* format and raw (no headers) streams are supported. Multiple compression
* algorithms (filters) are supported. Currently LZMA2 is the primary filter.
*
* liblzma is part of XZ Utils <https://xz.tukaani.org/xz-utils/>. XZ Utils
* includes a gzip-like command line tool named xz and some other tools.
* XZ Utils is developed and maintained by Lasse Collin and Jia Tan.
* liblzma is part of XZ Utils <https://tukaani.org/xz/>. XZ Utils includes
* a gzip-like command line tool named xz and some other tools. XZ Utils
* is developed and maintained by Lasse Collin and Jia Tan.
*
* Major parts of liblzma are based on code written by Igor Pavlov,
* specifically the LZMA SDK <https://7-zip.org/sdk.html>.
* Major parts of liblzma are based on Igor Pavlov's public domain LZMA SDK
* <https://7-zip.org/sdk.html>.
*
* The SHA-256 implementation in liblzma is based on code written by
* Wei Dai in Crypto++ Library <https://www.cryptopp.com/>.
*
* liblzma is distributed under the BSD Zero Clause License (0BSD).
* The SHA-256 implementation is based on the public domain code found from
* 7-Zip <https://7-zip.org/>, which has a modified version of the public
* domain SHA-256 code found from Crypto++ <https://www.cryptopp.com/>.
* The SHA-256 code in Crypto++ was written by Kevin Springle and Wei Dai.
*/
/*
* Author: Lasse Collin
*
* This file has been put into the public domain.
* You can do whatever you want with this file.
*/
#ifndef LZMA_H

View file

@ -1,5 +1,3 @@
/* SPDX-License-Identifier: 0BSD */
/**
* \file lzma/base.h
* \brief Data types and functions used in many places in liblzma API
@ -8,6 +6,9 @@
/*
* Author: Lasse Collin
*
* This file has been put into the public domain.
* You can do whatever you want with this file.
*/
#ifndef LZMA_H_INTERNAL
@ -20,8 +21,8 @@
*
* This is here because C89 doesn't have stdbool.h. To set a value for
* variables having type lzma_bool, you can use
* - C99's 'true' and 'false' from stdbool.h;
* - C++'s internal 'true' and 'false'; or
* - C99's `true' and `false' from stdbool.h;
* - C++'s internal `true' and `false'; or
* - integers one (true) and zero (false).
*/
typedef unsigned char lzma_bool;
@ -272,13 +273,13 @@ typedef enum {
/**
* \brief The 'action' argument for lzma_code()
* \brief The `action' argument for lzma_code()
*
* After the first use of LZMA_SYNC_FLUSH, LZMA_FULL_FLUSH, LZMA_FULL_BARRIER,
* or LZMA_FINISH, the same 'action' must be used until lzma_code() returns
* or LZMA_FINISH, the same `action' must be used until lzma_code() returns
* LZMA_STREAM_END. Also, the amount of input (that is, strm->avail_in) must
* not be modified by the application until lzma_code() returns
* LZMA_STREAM_END. Changing the 'action' or modifying the amount of input
* LZMA_STREAM_END. Changing the `action' or modifying the amount of input
* will make lzma_code() return LZMA_PROG_ERROR.
*/
typedef enum {
@ -392,8 +393,8 @@ typedef enum {
* Single-threaded mode only: liblzma doesn't make an internal copy of
* lzma_allocator. Thus, it is OK to change these function pointers in
* the middle of the coding process, but obviously it must be done
* carefully to make sure that the replacement 'free' can deallocate
* memory allocated by the earlier 'alloc' function(s).
* carefully to make sure that the replacement `free' can deallocate
* memory allocated by the earlier `alloc' function(s).
*
* Multithreaded mode: liblzma might internally store pointers to the
* lzma_allocator given via the lzma_stream structure. The application
@ -421,7 +422,7 @@ typedef struct {
* liblzma never sets this to zero.
*
* \return Pointer to the beginning of a memory block of
* 'size' bytes, or NULL if allocation fails
* `size' bytes, or NULL if allocation fails
* for some reason. When allocation fails, functions
* of liblzma return LZMA_MEM_ERROR.
*
@ -621,7 +622,7 @@ typedef struct {
* to and get output from liblzma.
*
* See the description of the coder-specific initialization function to find
* out what 'action' values are supported by the coder.
* out what `action' values are supported by the coder.
*
* \param strm Pointer to lzma_stream that is at least initialized
* with LZMA_STREAM_INIT.

View file

@ -1,5 +1,3 @@
/* SPDX-License-Identifier: 0BSD */
/**
* \file lzma/bcj.h
* \brief Branch/Call/Jump conversion filters
@ -8,6 +6,9 @@
/*
* Author: Lasse Collin
*
* This file has been put into the public domain.
* You can do whatever you want with this file.
*/
#ifndef LZMA_H_INTERNAL
@ -52,11 +53,6 @@
*/
#define LZMA_FILTER_ARM64 LZMA_VLI_C(0x0A)
/**
* \brief Filter for RISC-V binaries
*/
#define LZMA_FILTER_RISCV LZMA_VLI_C(0x0B)
/**
* \brief Options for BCJ filters

View file

@ -1,5 +1,3 @@
/* SPDX-License-Identifier: 0BSD */
/**
* \file lzma/block.h
* \brief .xz Block handling
@ -8,6 +6,9 @@
/*
* Author: Lasse Collin
*
* This file has been put into the public domain.
* You can do whatever you want with this file.
*/
#ifndef LZMA_H_INTERNAL

View file

@ -1,5 +1,3 @@
/* SPDX-License-Identifier: 0BSD */
/**
* \file lzma/check.h
* \brief Integrity checks
@ -8,6 +6,9 @@
/*
* Author: Lasse Collin
*
* This file has been put into the public domain.
* You can do whatever you want with this file.
*/
#ifndef LZMA_H_INTERNAL

View file

@ -1,5 +1,3 @@
/* SPDX-License-Identifier: 0BSD */
/**
* \file lzma/container.h
* \brief File formats
@ -8,6 +6,9 @@
/*
* Author: Lasse Collin
*
* This file has been put into the public domain.
* You can do whatever you want with this file.
*/
#ifndef LZMA_H_INTERNAL
@ -296,7 +297,7 @@ extern LZMA_API(uint64_t) lzma_easy_decoder_memusage(uint32_t preset)
* to call lzma_end() after failed initialization.
*
* If initialization succeeds, use lzma_code() to do the actual encoding.
* Valid values for 'action' (the second argument of lzma_code()) are
* Valid values for `action' (the second argument of lzma_code()) are
* LZMA_RUN, LZMA_SYNC_FLUSH, LZMA_FULL_FLUSH, and LZMA_FINISH. In future,
* there may be compression levels or flags that don't support LZMA_SYNC_FLUSH.
*
@ -434,34 +435,6 @@ extern LZMA_API(lzma_ret) lzma_stream_encoder_mt(
lzma_nothrow lzma_attr_warn_unused_result;
/**
* \brief Calculate recommended Block size for multithreaded .xz encoder
*
* This calculates a recommended Block size for multithreaded encoding given
* a filter chain. This is used internally by lzma_stream_encoder_mt() to
* determine the Block size if the block_size member is not set to the
* special value of 0 in the lzma_mt options struct.
*
* If one wishes to change the filters between Blocks, this function is
* helpful to set the block_size member of the lzma_mt struct before calling
* lzma_stream_encoder_mt(). Since the block_size member represents the
* maximum possible Block size for the multithreaded .xz encoder, one can
* use this function to find the maximum recommended Block size based on
* all planned filter chains. Otherwise, the multithreaded encoder will
* base its maximum Block size on the first filter chain used (if the
* block_size member is not set), which may unnecessarily limit the Block
* size for a later filter chain.
*
* \param filters Array of filters terminated with
* .id == LZMA_VLI_UNKNOWN.
*
* \return Recommended Block size in bytes, or UINT64_MAX if
* an error occurred.
*/
extern LZMA_API(uint64_t) lzma_mt_block_size(const lzma_filter *filters)
lzma_nothrow;
/**
* \brief Initialize .lzma encoder (legacy file format)
*
@ -678,13 +651,13 @@ extern LZMA_API(lzma_ret) lzma_microlzma_encoder(
* supported by liblzma, only the .xz and .lz formats allow concatenated
* files. Concatenated files are not allowed with the legacy .lzma format.
*
* This flag also affects the usage of the 'action' argument for lzma_code().
* This flag also affects the usage of the `action' argument for lzma_code().
* When LZMA_CONCATENATED is used, lzma_code() won't return LZMA_STREAM_END
* unless LZMA_FINISH is used as 'action'. Thus, the application has to set
* unless LZMA_FINISH is used as `action'. Thus, the application has to set
* LZMA_FINISH in the same way as it does when encoding.
*
* If LZMA_CONCATENATED is not used, the decoders still accept LZMA_FINISH
* as 'action' for lzma_code(), but the usage of LZMA_FINISH isn't required.
* as `action' for lzma_code(), but the usage of LZMA_FINISH isn't required.
*/
#define LZMA_CONCATENATED UINT32_C(0x08)
@ -818,7 +791,7 @@ extern LZMA_API(lzma_ret) lzma_auto_decoder(
/**
* \brief Initialize .lzma decoder (legacy file format)
*
* Valid 'action' arguments to lzma_code() are LZMA_RUN and LZMA_FINISH.
* Valid `action' arguments to lzma_code() are LZMA_RUN and LZMA_FINISH.
* There is no need to use LZMA_FINISH, but it's allowed because it may
* simplify certain types of applications.
*

View file

@ -1,5 +1,3 @@
/* SPDX-License-Identifier: 0BSD */
/**
* \file lzma/delta.h
* \brief Delta filter
@ -8,6 +6,9 @@
/*
* Author: Lasse Collin
*
* This file has been put into the public domain.
* You can do whatever you want with this file.
*/
#ifndef LZMA_H_INTERNAL

View file

@ -1,5 +1,3 @@
/* SPDX-License-Identifier: 0BSD */
/**
* \file lzma/filter.h
* \brief Common filter related types and functions
@ -8,6 +6,9 @@
/*
* Author: Lasse Collin
*
* This file has been put into the public domain.
* You can do whatever you want with this file.
*/
#ifndef LZMA_H_INTERNAL
@ -42,7 +43,7 @@ typedef struct {
/**
* \brief Filter ID
*
* Use constants whose name begin with 'LZMA_FILTER_' to specify
* Use constants whose name begin with `LZMA_FILTER_' to specify
* different filters. In an array of lzma_filter structures, use
* LZMA_VLI_UNKNOWN to indicate end of filters.
*
@ -198,7 +199,7 @@ extern LZMA_API(uint64_t) lzma_raw_decoder_memusage(const lzma_filter *filters)
*
* This function may be useful when implementing custom file formats.
*
* The 'action' with lzma_code() can be LZMA_RUN, LZMA_SYNC_FLUSH (if the
* The `action' with lzma_code() can be LZMA_RUN, LZMA_SYNC_FLUSH (if the
* filter chain supports it), or LZMA_FINISH.
*
* \param strm Pointer to lzma_stream that is at least
@ -222,7 +223,7 @@ extern LZMA_API(lzma_ret) lzma_raw_encoder(
*
* The initialization of raw decoder goes similarly to raw encoder.
*
* The 'action' with lzma_code() can be LZMA_RUN or LZMA_FINISH. Using
* The `action' with lzma_code() can be LZMA_RUN or LZMA_FINISH. Using
* LZMA_FINISH is not required, it is supported just for convenience.
*
* \param strm Pointer to lzma_stream that is at least

View file

@ -1,5 +1,3 @@
/* SPDX-License-Identifier: 0BSD */
/**
* \file lzma/hardware.h
* \brief Hardware information
@ -25,6 +23,9 @@
/*
* Author: Lasse Collin
*
* This file has been put into the public domain.
* You can do whatever you want with this file.
*/
#ifndef LZMA_H_INTERNAL

View file

@ -1,5 +1,3 @@
/* SPDX-License-Identifier: 0BSD */
/**
* \file lzma/index.h
* \brief Handling of .xz Index and related information
@ -8,6 +6,9 @@
/*
* Author: Lasse Collin
*
* This file has been put into the public domain.
* You can do whatever you want with this file.
*/
#ifndef LZMA_H_INTERNAL
@ -300,28 +301,6 @@ typedef enum {
} lzma_index_iter_mode;
/**
* \brief Mask for return value from lzma_index_checks() for check none
*
* \note This and the other CHECK_MASK macros were added in 5.5.1alpha.
*/
#define LZMA_INDEX_CHECK_MASK_NONE (UINT32_C(1) << LZMA_CHECK_NONE)
/**
* \brief Mask for return value from lzma_index_checks() for check CRC32
*/
#define LZMA_INDEX_CHECK_MASK_CRC32 (UINT32_C(1) << LZMA_CHECK_CRC32)
/**
* \brief Mask for return value from lzma_index_checks() for check CRC64
*/
#define LZMA_INDEX_CHECK_MASK_CRC64 (UINT32_C(1) << LZMA_CHECK_CRC64)
/**
* \brief Mask for return value from lzma_index_checks() for check SHA256
*/
#define LZMA_INDEX_CHECK_MASK_SHA256 (UINT32_C(1) << LZMA_CHECK_SHA256)
/**
* \brief Calculate memory usage of lzma_index
*
@ -452,7 +431,6 @@ extern LZMA_API(lzma_ret) lzma_index_stream_flags(
* showing the Check types to the user.
*
* The bitmask is 1 << check_id, e.g. CRC32 is 1 << 1 and SHA-256 is 1 << 10.
* These masks are defined for convenience as LZMA_INDEX_CHECK_MASK_XXX
*
* \param i Pointer to lzma_index structure
*
@ -708,7 +686,7 @@ extern LZMA_API(lzma_index *) lzma_index_dup(
* \param strm Pointer to properly prepared lzma_stream
* \param i Pointer to lzma_index which should be encoded.
*
* The valid 'action' values for lzma_code() are LZMA_RUN and LZMA_FINISH.
* The valid `action' values for lzma_code() are LZMA_RUN and LZMA_FINISH.
* It is enough to use only one of them (you can choose freely).
*
* \return Possible lzma_ret values:
@ -737,7 +715,7 @@ extern LZMA_API(lzma_ret) lzma_index_encoder(
* don't allow 0 here and return LZMA_PROG_ERROR;
* later versions treat 0 as if 1 had been specified.
*
* Valid 'action' arguments to lzma_code() are LZMA_RUN and LZMA_FINISH.
* Valid `action' arguments to lzma_code() are LZMA_RUN and LZMA_FINISH.
* There is no need to use LZMA_FINISH, but it's allowed because it may
* simplify certain types of applications.
*
@ -841,10 +819,10 @@ extern LZMA_API(lzma_ret) lzma_index_buffer_decode(lzma_index **i,
* expect to see the same exact value for the same file if you change the
* input buffer size or switch to a different liblzma version.
*
* Valid 'action' arguments to lzma_code() are LZMA_RUN and LZMA_FINISH.
* Valid `action' arguments to lzma_code() are LZMA_RUN and LZMA_FINISH.
* You only need to use LZMA_RUN; LZMA_FINISH is only supported because it
* might be convenient for some applications. If you use LZMA_FINISH and if
* lzma_code() asks the application to seek, remember to reset 'action' back
* lzma_code() asks the application to seek, remember to reset `action' back
* to LZMA_RUN unless you hit the end of the file again.
*
* Possible return values from lzma_code():

View file

@ -1,5 +1,3 @@
/* SPDX-License-Identifier: 0BSD */
/**
* \file lzma/index_hash.h
* \brief Validate Index by using a hash function
@ -11,6 +9,9 @@
/*
* Author: Lasse Collin
*
* This file has been put into the public domain.
* You can do whatever you want with this file.
*/
#ifndef LZMA_H_INTERNAL

View file

@ -1,5 +1,3 @@
/* SPDX-License-Identifier: 0BSD */
/**
* \file lzma/lzma12.h
* \brief LZMA1 and LZMA2 filters
@ -8,6 +6,9 @@
/*
* Author: Lasse Collin
*
* This file has been put into the public domain.
* You can do whatever you want with this file.
*/
#ifndef LZMA_H_INTERNAL
@ -288,7 +289,7 @@ typedef struct {
* \brief Number of literal context bits
*
* How many of the highest bits of the previous uncompressed
* eight-bit byte (also known as 'literal') are taken into
* eight-bit byte (also known as `literal') are taken into
* account when predicting the bits of the next literal.
*
* E.g. in typical English text, an upper-case letter is

View file

@ -1,5 +1,3 @@
/* SPDX-License-Identifier: 0BSD */
/**
* \file lzma/stream_flags.h
* \brief .xz Stream Header and Stream Footer encoder and decoder
@ -8,6 +6,9 @@
/*
* Author: Lasse Collin
*
* This file has been put into the public domain.
* You can do whatever you want with this file.
*/
#ifndef LZMA_H_INTERNAL

View file

@ -1,5 +1,3 @@
/* SPDX-License-Identifier: 0BSD */
/**
* \file lzma/version.h
* \brief Version number
@ -8,6 +6,9 @@
/*
* Author: Lasse Collin
*
* This file has been put into the public domain.
* You can do whatever you want with this file.
*/
#ifndef LZMA_H_INTERNAL
@ -19,10 +20,10 @@
#define LZMA_VERSION_MAJOR 5
/** \brief Minor version number of the liblzma release. */
#define LZMA_VERSION_MINOR 6
#define LZMA_VERSION_MINOR 4
/** \brief Patch version number of the liblzma release. */
#define LZMA_VERSION_PATCH 0
#define LZMA_VERSION_PATCH 5
/**
* \brief Version stability marker

View file

@ -1,5 +1,3 @@
/* SPDX-License-Identifier: 0BSD */
/**
* \file lzma/vli.h
* \brief Variable-length integer handling
@ -19,6 +17,9 @@
/*
* Author: Lasse Collin
*
* This file has been put into the public domain.
* You can do whatever you want with this file.
*/
#ifndef LZMA_H_INTERNAL

View file

@ -1,5 +1,3 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file check.c
@ -7,6 +5,9 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#include "check.h"

View file

@ -1,5 +1,3 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file check.h
@ -7,6 +5,9 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#ifndef LZMA_CHECK_H

View file

@ -1,119 +0,0 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file crc32_arm64.h
/// \brief CRC32 calculation with ARM64 optimization
//
// Authors: Chenxi Mao
// Jia Tan
// Hans Jansen
//
///////////////////////////////////////////////////////////////////////////////
#ifndef LZMA_CRC32_ARM64_H
#define LZMA_CRC32_ARM64_H
// MSVC always has the CRC intrinsics available when building for ARM64
// there is no need to include any header files.
#ifndef _MSC_VER
# include <arm_acle.h>
#endif
#if defined(CRC32_GENERIC) && defined(CRC32_ARCH_OPTIMIZED)
# if defined(HAVE_GETAUXVAL) || defined(HAVE_ELF_AUX_INFO)
# include <sys/auxv.h>
# elif defined(_WIN32)
# include <processthreadsapi.h>
# elif defined(__APPLE__) && defined(HAVE_SYSCTLBYNAME)
# include <sys/sysctl.h>
# endif
#endif
// Some EDG-based compilers support ARM64 and define __GNUC__
// (such as Nvidia's nvcc), but do not support function attributes.
//
// NOTE: Build systems check for this too, keep them in sync with this.
#if (defined(__GNUC__) || defined(__clang__)) && !defined(__EDG__)
# define crc_attr_target \
__attribute__((__target__("+crc")))
#else
# define crc_attr_target
#endif
crc_attr_target
static uint32_t
crc32_arch_optimized(const uint8_t *buf, size_t size, uint32_t crc)
{
crc = ~crc;
// Align the input buffer because this was shown to be
// significantly faster than unaligned accesses.
const size_t align_amount = my_min(size, (8 - (uintptr_t)buf) & 7);
for (const uint8_t *limit = buf + align_amount; buf < limit; ++buf)
crc = __crc32b(crc, *buf);
size -= align_amount;
// Process 8 bytes at a time. The end point is determined by
// ignoring the least significant three bits of size to ensure
// we do not process past the bounds of the buffer. This guarantees
// that limit is a multiple of 8 and is strictly less than size.
for (const uint8_t *limit = buf + (size & ~((size_t)7));
buf < limit; buf += 8)
crc = __crc32d(crc, aligned_read64le(buf));
// Process the remaining bytes that are not 8 byte aligned.
for (const uint8_t *limit = buf + (size & 7); buf < limit; ++buf)
crc = __crc32b(crc, *buf);
return ~crc;
}
#if defined(CRC32_GENERIC) && defined(CRC32_ARCH_OPTIMIZED)
static inline bool
is_arch_extension_supported(void)
{
#if defined(HAVE_GETAUXVAL)
return (getauxval(AT_HWCAP) & HWCAP_CRC32) != 0;
#elif defined(HAVE_ELF_AUX_INFO)
unsigned long feature_flags;
elf_aux_info(AT_HWCAP, &feature_flags, sizeof(feature_flags));
return feature_flags & HWCAP_CRC32 != 0;
#elif defined(_WIN32)
return IsProcessorFeaturePresent(
PF_ARM_V8_CRC32_INSTRUCTIONS_AVAILABLE);
#elif defined(__APPLE__) && defined(HAVE_SYSCTLBYNAME)
int has_crc32 = 0;
size_t size = sizeof(has_crc32);
// The sysctlbyname() function requires a string identifier for the
// CPU feature it tests. The Apple documentation lists the string
// "hw.optional.armv8_crc32", which can be found here:
// (https://developer.apple.com/documentation/kernel/1387446-sysctlbyname/determining_instruction_set_characteristics#3915619)
int err = sysctlbyname("hw.optional.armv8_crc32", &has_crc32,
&size, NULL, 0);
return !err && has_crc32;
#else
// If a runtime detection method cannot be found, then this must
// be a compile time error. The checks in crc_common.h should ensure
// a runtime detection method is always found if this function is
// built. It would be possible to just return false here, but this
// is inefficient for binary size and runtime since only the generic
// method could ever be used.
# error Runtime detection method unavailable.
#endif
}
#endif
#endif // LZMA_CRC32_ARM64_H

View file

@ -1,35 +1,30 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file crc32.c
/// \brief CRC32 calculation
///
/// Calculate the CRC32 using the slice-by-eight algorithm.
/// It is explained in this document:
/// http://www.intel.com/technology/comms/perfnet/download/CRC_generators.pdf
/// The code in this file is not the same as in Intel's paper, but
/// the basic principle is identical.
//
// Authors: Lasse Collin
// Ilya Kurdyukov
// Hans Jansen
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#include "check.h"
#include "crc_common.h"
#if defined(CRC_X86_CLMUL)
# define BUILDING_CRC32_CLMUL
# include "crc_x86_clmul.h"
#elif defined(CRC32_ARM64)
# include "crc32_arm64.h"
#endif
#include "crc_macros.h"
#ifdef CRC32_GENERIC
///////////////////
// Generic CRC32 //
///////////////////
static uint32_t
crc32_generic(const uint8_t *buf, size_t size, uint32_t crc)
// If you make any changes, do some benchmarking! Seemingly unrelated
// changes can very easily ruin the performance (and very probably is
// very compiler dependent).
extern LZMA_API(uint32_t)
lzma_crc32(const uint8_t *buf, size_t size, uint32_t crc)
{
crc = ~crc;
@ -85,153 +80,3 @@ crc32_generic(const uint8_t *buf, size_t size, uint32_t crc)
return ~crc;
}
#endif
#if defined(CRC32_GENERIC) && defined(CRC32_ARCH_OPTIMIZED)
//////////////////////////
// Function dispatching //
//////////////////////////
// If both the generic and arch-optimized implementations are built, then
// the function to use is selected at runtime because the system running
// the binary might not have the arch-specific instruction set extension(s)
// available. The three dispatch methods in order of priority:
//
// 1. Indirect function (ifunc). This method is slightly more efficient
// than the constructor method because it will change the entry in the
// Procedure Linkage Table (PLT) for the function either at load time or
// at the first call. This avoids having to call the function through a
// function pointer and will treat the function call like a regular call
// through the PLT. ifuncs are created by using
// __attribute__((__ifunc__("resolver"))) on a function which has no
// body. The "resolver" is the name of the function that chooses at
// runtime which implementation to use.
//
// 2. Constructor. This method uses __attribute__((__constructor__)) to
// set crc32_func at load time. This avoids extra computation (and any
// unlikely threading bugs) on the first call to lzma_crc32() to decide
// which implementation should be used.
//
// 3. First Call Resolution. On the very first call to lzma_crc32(), the
// call will be directed to crc32_dispatch() instead. This will set the
// appropriate implementation function and will not be called again.
// This method does not use any kind of locking but is safe because if
// multiple threads run the dispatcher simultaneously then they will all
// set crc32_func to the same value.
typedef uint32_t (*crc32_func_type)(
const uint8_t *buf, size_t size, uint32_t crc);
// Clang 16.0.0 and older has a bug where it marks the ifunc resolver
// function as unused since it is static and never used outside of
// __attribute__((__ifunc__())).
#if defined(CRC_USE_IFUNC) && defined(__clang__)
# pragma GCC diagnostic push
# pragma GCC diagnostic ignored "-Wunused-function"
#endif
// This resolver is shared between all three dispatch methods. It serves as
// the ifunc resolver if ifunc is supported, otherwise it is called as a
// regular function by the constructor or first call resolution methods.
static crc32_func_type
crc32_resolve(void)
{
return is_arch_extension_supported()
? &crc32_arch_optimized : &crc32_generic;
}
#if defined(CRC_USE_IFUNC) && defined(__clang__)
# pragma GCC diagnostic pop
#endif
#ifndef CRC_USE_IFUNC
#ifdef HAVE_FUNC_ATTRIBUTE_CONSTRUCTOR
// Constructor method.
# define CRC32_SET_FUNC_ATTR __attribute__((__constructor__))
static crc32_func_type crc32_func;
#else
// First Call Resolution method.
# define CRC32_SET_FUNC_ATTR
static uint32_t crc32_dispatch(const uint8_t *buf, size_t size, uint32_t crc);
static crc32_func_type crc32_func = &crc32_dispatch;
#endif
CRC32_SET_FUNC_ATTR
static void
crc32_set_func(void)
{
crc32_func = crc32_resolve();
return;
}
#ifndef HAVE_FUNC_ATTRIBUTE_CONSTRUCTOR
static uint32_t
crc32_dispatch(const uint8_t *buf, size_t size, uint32_t crc)
{
// When __attribute__((__ifunc__(...))) and
// __attribute__((__constructor__)) isn't supported, set the
// function pointer without any locking. If multiple threads run
// the detection code in parallel, they will all end up setting
// the pointer to the same value. This avoids the use of
// mythread_once() on every call to lzma_crc32() but this likely
// isn't strictly standards compliant. Let's change it if it breaks.
crc32_set_func();
return crc32_func(buf, size, crc);
}
#endif
#endif
#endif
#ifdef CRC_USE_IFUNC
extern LZMA_API(uint32_t)
lzma_crc32(const uint8_t *buf, size_t size, uint32_t crc)
__attribute__((__ifunc__("crc32_resolve")));
#else
extern LZMA_API(uint32_t)
lzma_crc32(const uint8_t *buf, size_t size, uint32_t crc)
{
#if defined(CRC32_GENERIC) && defined(CRC32_ARCH_OPTIMIZED)
// On x86-64, if CLMUL is available, it is the best for non-tiny
// inputs, being over twice as fast as the generic slice-by-four
// version. However, for size <= 16 it's different. In the extreme
// case of size == 1 the generic version can be five times faster.
// At size >= 8 the CLMUL starts to become reasonable. It
// varies depending on the alignment of buf too.
//
// The above doesn't include the overhead of mythread_once().
// At least on x86-64 GNU/Linux, pthread_once() is very fast but
// it still makes lzma_crc32(buf, 1, crc) 50-100 % slower. When
// size reaches 12-16 bytes the overhead becomes negligible.
//
// So using the generic version for size <= 16 may give better
// performance with tiny inputs but if such inputs happen rarely
// it's not so obvious because then the lookup table of the
// generic version may not be in the processor cache.
#ifdef CRC_USE_GENERIC_FOR_SMALL_INPUTS
if (size <= 16)
return crc32_generic(buf, size, crc);
#endif
/*
#ifndef HAVE_FUNC_ATTRIBUTE_CONSTRUCTOR
// See crc32_dispatch(). This would be the alternative which uses
// locking and doesn't use crc32_dispatch(). Note that on Windows
// this method needs Vista threads.
mythread_once(crc64_set_func);
#endif
*/
return crc32_func(buf, size, crc);
#elif defined(CRC32_ARCH_OPTIMIZED)
return crc32_arch_optimized(buf, size, crc);
#else
return crc32_generic(buf, size, crc);
#endif
}
#endif

View file

@ -1,5 +1,3 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file crc32_small.c
@ -7,6 +5,9 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#include "check.h"

View file

@ -1,5 +1,3 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file crc32_table.c
@ -7,38 +5,18 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#include "common.h"
// FIXME: Compared to crc_common.h this has to check for __x86_64__ too
// so that in 32-bit builds crc32_x86.S won't break due to a missing table.
#if defined(HAVE_USABLE_CLMUL) && ((defined(__x86_64__) && defined(__SSSE3__) \
&& defined(__SSE4_1__) && defined(__PCLMUL__)) \
|| (defined(__e2k__) && __iset__ >= 6))
# define X86_CLMUL_NO_TABLE 1
#endif
#if defined(HAVE_ARM64_CRC32) \
&& !defined(WORDS_BIGENDIAN) \
&& defined(__ARM_FEATURE_CRC32)
# define ARM64_CRC32_NO_TABLE 1
#endif
#if !defined(HAVE_ENCODERS) && (defined(X86_CLMUL_NO_TABLE) \
|| defined(ARM64_CRC32_NO_TABLE_))
// No table needed. Use a typedef to avoid an empty translation unit.
typedef void lzma_crc32_dummy;
#else
// Having the declaration here silences clang -Wmissing-variable-declarations.
extern const uint32_t lzma_crc32_table[8][256];
# ifdef WORDS_BIGENDIAN
# include "crc32_table_be.h"
# else
# include "crc32_table_le.h"
# endif
#ifdef WORDS_BIGENDIAN
# include "crc32_table_be.h"
#else
# include "crc32_table_le.h"
#endif

View file

@ -1,6 +1,4 @@
// SPDX-License-Identifier: 0BSD
// This file has been generated by crc32_tablegen.c.
/* This file has been automatically generated by crc32_tablegen.c. */
const uint32_t lzma_crc32_table[8][256] = {
{

View file

@ -1,6 +1,4 @@
// SPDX-License-Identifier: 0BSD
// This file has been generated by crc32_tablegen.c.
/* This file has been automatically generated by crc32_tablegen.c. */
const uint32_t lzma_crc32_table[8][256] = {
{

View file

@ -1,5 +1,3 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file crc32_tablegen.c
@ -11,6 +9,9 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#include <stdio.h>
@ -53,11 +54,9 @@ init_crc32_table(void)
static void
print_crc32_table(void)
{
// Split the SPDX string so that it won't accidentally match
// when tools search for the string.
printf("// SPDX" "-License-Identifier" ": 0BSD\n\n"
"// This file has been generated by crc32_tablegen.c.\n\n"
"const uint32_t lzma_crc32_table[8][256] = {\n\t{");
printf("/* This file has been automatically generated by "
"crc32_tablegen.c. */\n\n"
"const uint32_t lzma_crc32_table[8][256] = {\n\t{");
for (size_t s = 0; s < 8; ++s) {
for (size_t b = 0; b < 256; ++b) {
@ -83,11 +82,9 @@ print_crc32_table(void)
static void
print_lz_table(void)
{
// Split the SPDX string so that it won't accidentally match
// when tools search for the string.
printf("// SPDX" "-License-Identifier" ": 0BSD\n\n"
"// This file has been generated by crc32_tablegen.c.\n\n"
"const uint32_t lzma_lz_hash_table[256] = {");
printf("/* This file has been automatically generated by "
"crc32_tablegen.c. */\n\n"
"const uint32_t lzma_lz_hash_table[256] = {");
for (size_t b = 0; b < 256; ++b) {
if ((b % 4) == 0)

View file

@ -1,5 +1,3 @@
/* SPDX-License-Identifier: 0BSD */
/*
* Speed-optimized CRC32 using slicing-by-eight algorithm
*
@ -13,6 +11,9 @@
* Authors: Igor Pavlov (original version)
* Lasse Collin (AT&T syntax, PIC support, better portability)
*
* This file has been put into the public domain.
* You can do whatever you want with this file.
*
* This code needs lzma_crc32_table, which can be created using the
* following C code:

View file

@ -1,30 +1,85 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file crc64.c
/// \brief CRC64 calculation
///
/// There are two methods in this file. crc64_generic uses the
/// the slice-by-four algorithm. This is the same idea that is
/// used in crc32_fast.c, but for CRC64 we use only four tables
/// instead of eight to avoid increasing CPU cache usage.
///
/// crc64_clmul uses 32/64-bit x86 SSSE3, SSE4.1, and CLMUL instructions.
/// It was derived from
/// https://www.intel.com/content/dam/www/public/us/en/documents/white-papers/fast-crc-computation-generic-polynomials-pclmulqdq-paper.pdf
/// and the public domain code from https://github.com/rawrunprotected/crc
/// (URLs were checked on 2022-11-07).
///
/// FIXME: Builds for 32-bit x86 use crc64_x86.S by default instead
/// of this file and thus CLMUL version isn't available on 32-bit x86
/// unless configured with --disable-assembler. Even then the lookup table
/// isn't omitted in crc64_table.c since it doesn't know that assembly
/// code has been disabled.
//
// Authors: Lasse Collin
// Ilya Kurdyukov
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#include "check.h"
#include "crc_common.h"
#if defined(CRC_X86_CLMUL)
# define BUILDING_CRC64_CLMUL
# include "crc_x86_clmul.h"
#undef CRC_GENERIC
#undef CRC_CLMUL
#undef CRC_USE_GENERIC_FOR_SMALL_INPUTS
// If CLMUL cannot be used then only the generic slice-by-four is built.
#if !defined(HAVE_USABLE_CLMUL)
# define CRC_GENERIC 1
// If CLMUL is allowed unconditionally in the compiler options then the
// generic version can be omitted. Note that this doesn't work with MSVC
// as I don't know how to detect the features here.
//
// NOTE: Keep this this in sync with crc64_table.c.
#elif (defined(__SSSE3__) && defined(__SSE4_1__) && defined(__PCLMUL__)) \
|| (defined(__e2k__) && __iset__ >= 6)
# define CRC_CLMUL 1
// Otherwise build both and detect at runtime which version to use.
#else
# define CRC_GENERIC 1
# define CRC_CLMUL 1
/*
// The generic code is much faster with 1-8-byte inputs and has
// similar performance up to 16 bytes at least in microbenchmarks
// (it depends on input buffer alignment too). If both versions are
// built, this #define will use the generic version for inputs up to
// 16 bytes and CLMUL for bigger inputs. It saves a little in code
// size since the special cases for 0-16-byte inputs will be omitted
// from the CLMUL code.
# define CRC_USE_GENERIC_FOR_SMALL_INPUTS 1
*/
# if defined(_MSC_VER)
# include <intrin.h>
# elif defined(HAVE_CPUID_H)
# include <cpuid.h>
# endif
#endif
#ifdef CRC64_GENERIC
/////////////////////////////////
// Generic slice-by-four CRC64 //
/////////////////////////////////
#ifdef CRC_GENERIC
#include "crc_macros.h"
#ifdef WORDS_BIGENDIAN
# define A1(x) ((x) >> 56)
#else
@ -81,51 +136,336 @@ crc64_generic(const uint8_t *buf, size_t size, uint64_t crc)
#endif
#if defined(CRC64_GENERIC) && defined(CRC64_ARCH_OPTIMIZED)
/////////////////////
// x86 CLMUL CRC64 //
/////////////////////
//////////////////////////
// Function dispatching //
//////////////////////////
#ifdef CRC_CLMUL
// If both the generic and arch-optimized implementations are usable, then
// the function that is used is selected at runtime. See crc32_fast.c.
#include <immintrin.h>
typedef uint64_t (*crc64_func_type)(
const uint8_t *buf, size_t size, uint64_t crc);
#if defined(CRC_USE_IFUNC) && defined(__clang__)
# pragma GCC diagnostic push
# pragma GCC diagnostic ignored "-Wunused-function"
#endif
static crc64_func_type
crc64_resolve(void)
/*
// These functions were used to generate the constants
// at the top of crc64_clmul().
static uint64_t
calc_lo(uint64_t poly)
{
return is_arch_extension_supported()
? &crc64_arch_optimized : &crc64_generic;
uint64_t a = poly;
uint64_t b = 0;
for (unsigned i = 0; i < 64; ++i) {
b = (b >> 1) | (a << 63);
a = (a >> 1) ^ (a & 1 ? poly : 0);
}
return b;
}
#if defined(CRC_USE_IFUNC) && defined(__clang__)
static uint64_t
calc_hi(uint64_t poly, uint64_t a)
{
for (unsigned i = 0; i < 64; ++i)
a = (a >> 1) ^ (a & 1 ? poly : 0);
return a;
}
*/
#define MASK_L(in, mask, r) \
r = _mm_shuffle_epi8(in, mask)
#define MASK_H(in, mask, r) \
r = _mm_shuffle_epi8(in, _mm_xor_si128(mask, vsign))
#define MASK_LH(in, mask, low, high) \
MASK_L(in, mask, low); \
MASK_H(in, mask, high)
// MSVC (VS2015 - VS2022) produces bad 32-bit x86 code from the CLMUL CRC
// code when optimizations are enabled (release build). According to the bug
// report, the ebx register is corrupted and the calculated result is wrong.
// Trying to workaround the problem with "__asm mov ebx, ebx" didn't help.
// The following pragma works and performance is still good. x86-64 builds
// aren't affected by this problem.
//
// NOTE: Another pragma after the function restores the optimizations.
// If the #if condition here is updated, the other one must be updated too.
#if defined(_MSC_VER) && !defined(__INTEL_COMPILER) && !defined(__clang__) \
&& defined(_M_IX86)
# pragma optimize("g", off)
#endif
// EDG-based compilers (Intel's classic compiler and compiler for E2K) can
// define __GNUC__ but the attribute must not be used with them.
// The new Clang-based ICX needs the attribute.
//
// NOTE: Build systems check for this too, keep them in sync with this.
#if (defined(__GNUC__) || defined(__clang__)) && !defined(__EDG__)
__attribute__((__target__("ssse3,sse4.1,pclmul")))
#endif
// The intrinsics use 16-byte-aligned reads from buf, thus they may read
// up to 15 bytes before or after the buffer (depending on the alignment
// of the buf argument). The values of the extra bytes are ignored.
// This unavoidably trips -fsanitize=address so address sanitizier has
// to be disabled for this function.
#if lzma_has_attribute(__no_sanitize_address__)
__attribute__((__no_sanitize_address__))
#endif
static uint64_t
crc64_clmul(const uint8_t *buf, size_t size, uint64_t crc)
{
// The prototypes of the intrinsics use signed types while most of
// the values are treated as unsigned here. These warnings in this
// function have been checked and found to be harmless so silence them.
#if TUKLIB_GNUC_REQ(4, 6) || defined(__clang__)
# pragma GCC diagnostic push
# pragma GCC diagnostic ignored "-Wsign-conversion"
# pragma GCC diagnostic ignored "-Wconversion"
#endif
#ifndef CRC_USE_GENERIC_FOR_SMALL_INPUTS
// The code assumes that there is at least one byte of input.
if (size == 0)
return crc;
#endif
// const uint64_t poly = 0xc96c5795d7870f42; // CRC polynomial
const uint64_t p = 0x92d8af2baf0e1e85; // (poly << 1) | 1
const uint64_t mu = 0x9c3e466c172963d5; // (calc_lo(poly) << 1) | 1
const uint64_t k2 = 0xdabe95afc7875f40; // calc_hi(poly, 1)
const uint64_t k1 = 0xe05dd497ca393ae4; // calc_hi(poly, k2)
const __m128i vfold0 = _mm_set_epi64x(p, mu);
const __m128i vfold1 = _mm_set_epi64x(k2, k1);
// Create a vector with 8-bit values 0 to 15. This is used to
// construct control masks for _mm_blendv_epi8 and _mm_shuffle_epi8.
const __m128i vramp = _mm_setr_epi32(
0x03020100, 0x07060504, 0x0b0a0908, 0x0f0e0d0c);
// This is used to inverse the control mask of _mm_shuffle_epi8
// so that bytes that wouldn't be picked with the original mask
// will be picked and vice versa.
const __m128i vsign = _mm_set1_epi8(0x80);
// Memory addresses A to D and the distances between them:
//
// A B C D
// [skip_start][size][skip_end]
// [ size2 ]
//
// A and D are 16-byte aligned. B and C are 1-byte aligned.
// skip_start and skip_end are 0-15 bytes. size is at least 1 byte.
//
// A = aligned_buf will initially point to this address.
// B = The address pointed by the caller-supplied buf.
// C = buf + size == aligned_buf + size2
// D = buf + size + skip_end == aligned_buf + size2 + skip_end
const size_t skip_start = (size_t)((uintptr_t)buf & 15);
const size_t skip_end = (size_t)((0U - (uintptr_t)(buf + size)) & 15);
const __m128i *aligned_buf = (const __m128i *)(
(uintptr_t)buf & ~(uintptr_t)15);
// If size2 <= 16 then the whole input fits into a single 16-byte
// vector. If size2 > 16 then at least two 16-byte vectors must
// be processed. If size2 > 16 && size <= 16 then there is only
// one 16-byte vector's worth of input but it is unaligned in memory.
//
// NOTE: There is no integer overflow here if the arguments are valid.
// If this overflowed, buf + size would too.
size_t size2 = skip_start + size;
// Masks to be used with _mm_blendv_epi8 and _mm_shuffle_epi8:
// The first skip_start or skip_end bytes in the vectors will have
// the high bit (0x80) set. _mm_blendv_epi8 and _mm_shuffle_epi8
// will produce zeros for these positions. (Bitwise-xor of these
// masks with vsign will produce the opposite behavior.)
const __m128i mask_start
= _mm_sub_epi8(vramp, _mm_set1_epi8(skip_start));
const __m128i mask_end = _mm_sub_epi8(vramp, _mm_set1_epi8(skip_end));
// Get the first 1-16 bytes into data0. If loading less than 16 bytes,
// the bytes are loaded to the high bits of the vector and the least
// significant positions are filled with zeros.
const __m128i data0 = _mm_blendv_epi8(_mm_load_si128(aligned_buf),
_mm_setzero_si128(), mask_start);
++aligned_buf;
#if defined(__i386__) || defined(_M_IX86)
const __m128i initial_crc = _mm_set_epi64x(0, ~crc);
#else
// GCC and Clang would produce good code with _mm_set_epi64x
// but MSVC needs _mm_cvtsi64_si128 on x86-64.
const __m128i initial_crc = _mm_cvtsi64_si128(~crc);
#endif
__m128i v0, v1, v2, v3;
#ifndef CRC_USE_GENERIC_FOR_SMALL_INPUTS
if (size <= 16) {
// Right-shift initial_crc by 1-16 bytes based on "size"
// and store the result in v1 (high bytes) and v0 (low bytes).
//
// NOTE: The highest 8 bytes of initial_crc are zeros so
// v1 will be filled with zeros if size >= 8. The highest 8
// bytes of v1 will always become zeros.
//
// [ v1 ][ v0 ]
// [ initial_crc ] size == 1
// [ initial_crc ] size == 2
// [ initial_crc ] size == 15
// [ initial_crc ] size == 16 (all in v0)
const __m128i mask_low = _mm_add_epi8(
vramp, _mm_set1_epi8(size - 16));
MASK_LH(initial_crc, mask_low, v0, v1);
if (size2 <= 16) {
// There are 1-16 bytes of input and it is all
// in data0. Copy the input bytes to v3. If there
// are fewer than 16 bytes, the low bytes in v3
// will be filled with zeros. That is, the input
// bytes are stored to the same position as
// (part of) initial_crc is in v0.
MASK_L(data0, mask_end, v3);
} else {
// There are 2-16 bytes of input but not all bytes
// are in data0.
const __m128i data1 = _mm_load_si128(aligned_buf);
// Collect the 2-16 input bytes from data0 and data1
// to v2 and v3, and bitwise-xor them with the
// low bits of initial_crc in v0. Note that the
// the second xor is below this else-block as it
// is shared with the other branch.
MASK_H(data0, mask_end, v2);
MASK_L(data1, mask_end, v3);
v0 = _mm_xor_si128(v0, v2);
}
v0 = _mm_xor_si128(v0, v3);
v1 = _mm_alignr_epi8(v1, v0, 8);
} else
#endif
{
const __m128i data1 = _mm_load_si128(aligned_buf);
MASK_LH(initial_crc, mask_start, v0, v1);
v0 = _mm_xor_si128(v0, data0);
v1 = _mm_xor_si128(v1, data1);
#define FOLD \
v1 = _mm_xor_si128(v1, _mm_clmulepi64_si128(v0, vfold1, 0x00)); \
v0 = _mm_xor_si128(v1, _mm_clmulepi64_si128(v0, vfold1, 0x11));
while (size2 > 32) {
++aligned_buf;
size2 -= 16;
FOLD
v1 = _mm_load_si128(aligned_buf);
}
if (size2 < 32) {
MASK_H(v0, mask_end, v2);
MASK_L(v0, mask_end, v0);
MASK_L(v1, mask_end, v3);
v1 = _mm_or_si128(v2, v3);
}
FOLD
v1 = _mm_srli_si128(v0, 8);
#undef FOLD
}
v1 = _mm_xor_si128(_mm_clmulepi64_si128(v0, vfold1, 0x10), v1);
v0 = _mm_clmulepi64_si128(v1, vfold0, 0x00);
v2 = _mm_clmulepi64_si128(v0, vfold0, 0x10);
v0 = _mm_xor_si128(_mm_xor_si128(v2, _mm_slli_si128(v0, 8)), v1);
#if defined(__i386__) || defined(_M_IX86)
return ~(((uint64_t)(uint32_t)_mm_extract_epi32(v0, 3) << 32) |
(uint64_t)(uint32_t)_mm_extract_epi32(v0, 2));
#else
return ~(uint64_t)_mm_extract_epi64(v0, 1);
#endif
#if TUKLIB_GNUC_REQ(4, 6) || defined(__clang__)
# pragma GCC diagnostic pop
#endif
}
#if defined(_MSC_VER) && !defined(__INTEL_COMPILER) && !defined(__clang__) \
&& defined(_M_IX86)
# pragma optimize("", on)
#endif
#endif
////////////////////////
// Detect CPU support //
////////////////////////
#if defined(CRC_GENERIC) && defined(CRC_CLMUL)
static inline bool
is_clmul_supported(void)
{
int success = 1;
uint32_t r[4]; // eax, ebx, ecx, edx
#if defined(_MSC_VER)
// This needs <intrin.h> with MSVC. ICC has it as a built-in
// on all platforms.
__cpuid(r, 1);
#elif defined(HAVE_CPUID_H)
// Compared to just using __asm__ to run CPUID, this also checks
// that CPUID is supported and saves and restores ebx as that is
// needed with GCC < 5 with position-independent code (PIC).
success = __get_cpuid(1, &r[0], &r[1], &r[2], &r[3]);
#else
// Just a fallback that shouldn't be needed.
__asm__("cpuid\n\t"
: "=a"(r[0]), "=b"(r[1]), "=c"(r[2]), "=d"(r[3])
: "a"(1), "c"(0));
#endif
// Returns true if these are supported:
// CLMUL (bit 1 in ecx)
// SSSE3 (bit 9 in ecx)
// SSE4.1 (bit 19 in ecx)
const uint32_t ecx_mask = (1 << 1) | (1 << 9) | (1 << 19);
return success && (r[2] & ecx_mask) == ecx_mask;
// Alternative methods that weren't used:
// - ICC's _may_i_use_cpu_feature: the other methods should work too.
// - GCC >= 6 / Clang / ICX __builtin_cpu_supports("pclmul")
//
// CPUID decding is needed with MSVC anyway and older GCC. This keeps
// the feature checks in the build system simpler too. The nice thing
// about __builtin_cpu_supports would be that it generates very short
// code as is it only reads a variable set at startup but a few bytes
// doesn't matter here.
}
#ifndef CRC_USE_IFUNC
#ifdef HAVE_FUNC_ATTRIBUTE_CONSTRUCTOR
# define CRC64_FUNC_INIT
# define CRC64_SET_FUNC_ATTR __attribute__((__constructor__))
static crc64_func_type crc64_func;
#else
# define CRC64_FUNC_INIT = &crc64_dispatch
# define CRC64_SET_FUNC_ATTR
static uint64_t crc64_dispatch(const uint8_t *buf, size_t size, uint64_t crc);
static crc64_func_type crc64_func = &crc64_dispatch;
#endif
// Pointer to the the selected CRC64 method.
static uint64_t (*crc64_func)(const uint8_t *buf, size_t size, uint64_t crc)
CRC64_FUNC_INIT;
CRC64_SET_FUNC_ATTR
static void
crc64_set_func(void)
{
crc64_func = crc64_resolve();
crc64_func = is_clmul_supported() ? &crc64_clmul : &crc64_generic;
return;
}
@ -134,41 +474,65 @@ crc64_set_func(void)
static uint64_t
crc64_dispatch(const uint8_t *buf, size_t size, uint64_t crc)
{
// When __attribute__((__constructor__)) isn't supported, set the
// function pointer without any locking. If multiple threads run
// the detection code in parallel, they will all end up setting
// the pointer to the same value. This avoids the use of
// mythread_once() on every call to lzma_crc64() but this likely
// isn't strictly standards compliant. Let's change it if it breaks.
crc64_set_func();
return crc64_func(buf, size, crc);
}
#endif
#endif
#endif
#ifdef CRC_USE_IFUNC
extern LZMA_API(uint64_t)
lzma_crc64(const uint8_t *buf, size_t size, uint64_t crc)
__attribute__((__ifunc__("crc64_resolve")));
#else
extern LZMA_API(uint64_t)
lzma_crc64(const uint8_t *buf, size_t size, uint64_t crc)
{
#if defined(CRC64_GENERIC) && defined(CRC64_ARCH_OPTIMIZED)
#if defined(CRC_GENERIC) && defined(CRC_CLMUL)
// If CLMUL is available, it is the best for non-tiny inputs,
// being over twice as fast as the generic slice-by-four version.
// However, for size <= 16 it's different. In the extreme case
// of size == 1 the generic version can be five times faster.
// At size >= 8 the CLMUL starts to become reasonable. It
// varies depending on the alignment of buf too.
//
// The above doesn't include the overhead of mythread_once().
// At least on x86-64 GNU/Linux, pthread_once() is very fast but
// it still makes lzma_crc64(buf, 1, crc) 50-100 % slower. When
// size reaches 12-16 bytes the overhead becomes negligible.
//
// So using the generic version for size <= 16 may give better
// performance with tiny inputs but if such inputs happen rarely
// it's not so obvious because then the lookup table of the
// generic version may not be in the processor cache.
#ifdef CRC_USE_GENERIC_FOR_SMALL_INPUTS
if (size <= 16)
return crc64_generic(buf, size, crc);
#endif
/*
#ifndef HAVE_FUNC_ATTRIBUTE_CONSTRUCTOR
// See crc64_dispatch(). This would be the alternative which uses
// locking and doesn't use crc64_dispatch(). Note that on Windows
// this method needs Vista threads.
mythread_once(crc64_set_func);
#endif
*/
return crc64_func(buf, size, crc);
#elif defined(CRC64_ARCH_OPTIMIZED)
// If arch-optimized version is used unconditionally without runtime
// CPU detection then omitting the generic version and its 8 KiB
// lookup table makes the library smaller.
#elif defined(CRC_CLMUL)
// If CLMUL is used unconditionally without runtime CPU detection
// then omitting the generic version and its 8 KiB lookup table
// makes the library smaller.
//
// FIXME: Lookup table isn't currently omitted on 32-bit x86,
// see crc64_table.c.
return crc64_arch_optimized(buf, size, crc);
return crc64_clmul(buf, size, crc);
#else
return crc64_generic(buf, size, crc);
#endif
}
#endif

View file

@ -1,5 +1,3 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file crc64_small.c
@ -7,6 +5,9 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#include "check.h"

View file

@ -1,5 +1,3 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file crc64_table.c
@ -7,21 +5,19 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#include "common.h"
// FIXME: Compared to crc_common.h this has to check for __x86_64__ too
// FIXME: Compared to crc64_fast.c this has to check for __x86_64__ too
// so that in 32-bit builds crc64_x86.S won't break due to a missing table.
#if defined(HAVE_USABLE_CLMUL) && ((defined(__x86_64__) && defined(__SSSE3__) \
#if (defined(__x86_64__) && defined(__SSSE3__) \
&& defined(__SSE4_1__) && defined(__PCLMUL__)) \
|| (defined(__e2k__) && __iset__ >= 6))
# define X86_CLMUL_NO_TABLE 1
#endif
#ifdef X86_CLMUL_NO_TABLE
|| (defined(__e2k__) && __iset__ >= 6)
// No table needed. Use a typedef to avoid an empty translation unit.
typedef void lzma_crc64_dummy;

View file

@ -1,6 +1,4 @@
// SPDX-License-Identifier: 0BSD
// This file has been generated by crc64_tablegen.c.
/* This file has been automatically generated by crc64_tablegen.c. */
const uint64_t lzma_crc64_table[4][256] = {
{

View file

@ -1,6 +1,4 @@
// SPDX-License-Identifier: 0BSD
// This file has been generated by crc64_tablegen.c.
/* This file has been automatically generated by crc64_tablegen.c. */
const uint64_t lzma_crc64_table[4][256] = {
{

View file

@ -1,5 +1,3 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file crc64_tablegen.c
@ -10,6 +8,9 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#include <stdio.h>
@ -52,11 +53,9 @@ init_crc64_table(void)
static void
print_crc64_table(void)
{
// Split the SPDX string so that it won't accidentally match
// when tools search for the string.
printf("// SPDX" "-License-Identifier" ": 0BSD\n\n"
"// This file has been generated by crc64_tablegen.c.\n\n"
"const uint64_t lzma_crc64_table[4][256] = {\n\t{");
printf("/* This file has been automatically generated by "
"crc64_tablegen.c. */\n\n"
"const uint64_t lzma_crc64_table[4][256] = {\n\t{");
for (size_t s = 0; s < 4; ++s) {
for (size_t b = 0; b < 256; ++b) {

View file

@ -1,5 +1,3 @@
/* SPDX-License-Identifier: 0BSD */
/*
* Speed-optimized CRC64 using slicing-by-four algorithm
*
@ -9,6 +7,9 @@
* Authors: Igor Pavlov (original CRC32 assembly code)
* Lasse Collin (CRC64 adaptation of the modified CRC32 code)
*
* This file has been put into the public domain.
* You can do whatever you want with this file.
*
* This code needs lzma_crc64_table, which can be created using the
* following C code:

View file

@ -1,143 +0,0 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file crc_common.h
/// \brief Some functions and macros for CRC32 and CRC64
//
// Authors: Lasse Collin
// Ilya Kurdyukov
// Hans Jansen
// Jia Tan
//
///////////////////////////////////////////////////////////////////////////////
#ifndef LZMA_CRC_COMMON_H
#define LZMA_CRC_COMMON_H
#include "common.h"
#ifdef WORDS_BIGENDIAN
# define A(x) ((x) >> 24)
# define B(x) (((x) >> 16) & 0xFF)
# define C(x) (((x) >> 8) & 0xFF)
# define D(x) ((x) & 0xFF)
# define S8(x) ((x) << 8)
# define S32(x) ((x) << 32)
#else
# define A(x) ((x) & 0xFF)
# define B(x) (((x) >> 8) & 0xFF)
# define C(x) (((x) >> 16) & 0xFF)
# define D(x) ((x) >> 24)
# define S8(x) ((x) >> 8)
# define S32(x) ((x) >> 32)
#endif
// CRC CLMUL code needs this because accessing input buffers that aren't
// aligned to the vector size will inherently trip the address sanitizer.
#if lzma_has_attribute(__no_sanitize_address__)
# define crc_attr_no_sanitize_address \
__attribute__((__no_sanitize_address__))
#else
# define crc_attr_no_sanitize_address
#endif
// Keep this in sync with changes to crc32_arm64.h
#if defined(_WIN32) || defined(HAVE_GETAUXVAL) \
|| defined(HAVE_ELF_AUX_INFO) \
|| (defined(__APPLE__) && defined(HAVE_SYSCTLBYNAME))
# define ARM64_RUNTIME_DETECTION 1
#endif
#undef CRC32_GENERIC
#undef CRC64_GENERIC
#undef CRC32_ARCH_OPTIMIZED
#undef CRC64_ARCH_OPTIMIZED
// The x86 CLMUL is used for both CRC32 and CRC64.
#undef CRC_X86_CLMUL
#undef CRC32_ARM64
#undef CRC64_ARM64_CLMUL
#undef CRC_USE_IFUNC
#undef CRC_USE_GENERIC_FOR_SMALL_INPUTS
// ARM64 CRC32 instruction is only useful for CRC32. Currently, only
// little endian is supported since we were unable to test on a big
// endian machine.
//
// NOTE: Keep this and the next check in sync with the macro
// ARM64_CRC32_NO_TABLE in crc32_table.c
#if defined(HAVE_ARM64_CRC32) && !defined(WORDS_BIGENDIAN)
// Allow ARM64 CRC32 instruction without a runtime check if
// __ARM_FEATURE_CRC32 is defined. GCC and Clang only define this if the
// proper compiler options are used.
# if defined(__ARM_FEATURE_CRC32)
# define CRC32_ARCH_OPTIMIZED 1
# define CRC32_ARM64 1
# elif defined(ARM64_RUNTIME_DETECTION)
# define CRC32_ARCH_OPTIMIZED 1
# define CRC32_ARM64 1
# define CRC32_GENERIC 1
# endif
#endif
#if defined(HAVE_USABLE_CLMUL)
// If CLMUL is allowed unconditionally in the compiler options then the
// generic version can be omitted. Note that this doesn't work with MSVC
// as I don't know how to detect the features here.
//
// NOTE: Keep this in sync with the CLMUL_NO_TABLE macro in crc32_table.c.
# if (defined(__SSSE3__) && defined(__SSE4_1__) && defined(__PCLMUL__)) \
|| (defined(__e2k__) && __iset__ >= 6)
# define CRC32_ARCH_OPTIMIZED 1
# define CRC64_ARCH_OPTIMIZED 1
# define CRC_X86_CLMUL 1
# else
# define CRC32_GENERIC 1
# define CRC64_GENERIC 1
# define CRC32_ARCH_OPTIMIZED 1
# define CRC64_ARCH_OPTIMIZED 1
# define CRC_X86_CLMUL 1
# ifdef HAVE_FUNC_ATTRIBUTE_IFUNC
# define CRC_USE_IFUNC 1
# endif
/*
// The generic code is much faster with 1-8-byte inputs and
// has similar performance up to 16 bytes at least in
// microbenchmarks (it depends on input buffer alignment
// too). If both versions are built, this #define will use
// the generic version for inputs up to 16 bytes and CLMUL
// for bigger inputs. It saves a little in code size since
// the special cases for 0-16-byte inputs will be omitted
// from the CLMUL code.
# ifndef CRC_USE_IFUNC
# define CRC_USE_GENERIC_FOR_SMALL_INPUTS 1
# endif
*/
# endif
#endif
// For CRC32 use the generic slice-by-eight implementation if no optimized
// version is available.
#if !defined(CRC32_ARCH_OPTIMIZED) && !defined(CRC32_GENERIC)
# define CRC32_GENERIC 1
#endif
// For CRC64 use the generic slice-by-four implementation if no optimized
// version is available.
#if !defined(CRC64_ARCH_OPTIMIZED) && !defined(CRC64_GENERIC)
# define CRC64_GENERIC 1
#endif
#endif

View file

@ -0,0 +1,30 @@
///////////////////////////////////////////////////////////////////////////////
//
/// \file crc_macros.h
/// \brief Some endian-dependent macros for CRC32 and CRC64
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#ifdef WORDS_BIGENDIAN
# define A(x) ((x) >> 24)
# define B(x) (((x) >> 16) & 0xFF)
# define C(x) (((x) >> 8) & 0xFF)
# define D(x) ((x) & 0xFF)
# define S8(x) ((x) << 8)
# define S32(x) ((x) << 32)
#else
# define A(x) ((x) & 0xFF)
# define B(x) (((x) >> 8) & 0xFF)
# define C(x) (((x) >> 16) & 0xFF)
# define D(x) ((x) >> 24)
# define S8(x) ((x) >> 8)
# define S32(x) ((x) >> 32)
#endif

View file

@ -1,435 +0,0 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file crc_x86_clmul.h
/// \brief CRC32 and CRC64 implementations using CLMUL instructions.
///
/// The CRC32 and CRC64 implementations use 32/64-bit x86 SSSE3, SSE4.1, and
/// CLMUL instructions. This is compatible with Elbrus 2000 (E2K) too.
///
/// They were derived from
/// https://www.researchgate.net/publication/263424619_Fast_CRC_computation
/// and the public domain code from https://github.com/rawrunprotected/crc
/// (URLs were checked on 2023-10-14).
///
/// While this file has both CRC32 and CRC64 implementations, only one
/// should be built at a time to ensure that crc_simd_body() is inlined
/// even with compilers with which lzma_always_inline expands to plain inline.
/// The version to build is selected by defining BUILDING_CRC32_CLMUL or
/// BUILDING_CRC64_CLMUL before including this file.
///
/// FIXME: Builds for 32-bit x86 use the assembly .S files by default
/// unless configured with --disable-assembler. Even then the lookup table
/// isn't omitted in crc64_table.c since it doesn't know that assembly
/// code has been disabled.
//
// Authors: Ilya Kurdyukov
// Hans Jansen
// Lasse Collin
// Jia Tan
//
///////////////////////////////////////////////////////////////////////////////
// This file must not be included more than once.
#ifdef LZMA_CRC_X86_CLMUL_H
# error crc_x86_clmul.h was included twice.
#endif
#define LZMA_CRC_X86_CLMUL_H
#include <immintrin.h>
#if defined(_MSC_VER)
# include <intrin.h>
#elif defined(HAVE_CPUID_H)
# include <cpuid.h>
#endif
// EDG-based compilers (Intel's classic compiler and compiler for E2K) can
// define __GNUC__ but the attribute must not be used with them.
// The new Clang-based ICX needs the attribute.
//
// NOTE: Build systems check for this too, keep them in sync with this.
#if (defined(__GNUC__) || defined(__clang__)) && !defined(__EDG__)
# define crc_attr_target \
__attribute__((__target__("ssse3,sse4.1,pclmul")))
#else
# define crc_attr_target
#endif
#define MASK_L(in, mask, r) r = _mm_shuffle_epi8(in, mask)
#define MASK_H(in, mask, r) \
r = _mm_shuffle_epi8(in, _mm_xor_si128(mask, vsign))
#define MASK_LH(in, mask, low, high) \
MASK_L(in, mask, low); \
MASK_H(in, mask, high)
crc_attr_target
crc_attr_no_sanitize_address
static lzma_always_inline void
crc_simd_body(const uint8_t *buf, const size_t size, __m128i *v0, __m128i *v1,
const __m128i vfold16, const __m128i initial_crc)
{
// Create a vector with 8-bit values 0 to 15. This is used to
// construct control masks for _mm_blendv_epi8 and _mm_shuffle_epi8.
const __m128i vramp = _mm_setr_epi32(
0x03020100, 0x07060504, 0x0b0a0908, 0x0f0e0d0c);
// This is used to inverse the control mask of _mm_shuffle_epi8
// so that bytes that wouldn't be picked with the original mask
// will be picked and vice versa.
const __m128i vsign = _mm_set1_epi8(-0x80);
// Memory addresses A to D and the distances between them:
//
// A B C D
// [skip_start][size][skip_end]
// [ size2 ]
//
// A and D are 16-byte aligned. B and C are 1-byte aligned.
// skip_start and skip_end are 0-15 bytes. size is at least 1 byte.
//
// A = aligned_buf will initially point to this address.
// B = The address pointed by the caller-supplied buf.
// C = buf + size == aligned_buf + size2
// D = buf + size + skip_end == aligned_buf + size2 + skip_end
const size_t skip_start = (size_t)((uintptr_t)buf & 15);
const size_t skip_end = (size_t)((0U - (uintptr_t)(buf + size)) & 15);
const __m128i *aligned_buf = (const __m128i *)(
(uintptr_t)buf & ~(uintptr_t)15);
// If size2 <= 16 then the whole input fits into a single 16-byte
// vector. If size2 > 16 then at least two 16-byte vectors must
// be processed. If size2 > 16 && size <= 16 then there is only
// one 16-byte vector's worth of input but it is unaligned in memory.
//
// NOTE: There is no integer overflow here if the arguments
// are valid. If this overflowed, buf + size would too.
const size_t size2 = skip_start + size;
// Masks to be used with _mm_blendv_epi8 and _mm_shuffle_epi8:
// The first skip_start or skip_end bytes in the vectors will have
// the high bit (0x80) set. _mm_blendv_epi8 and _mm_shuffle_epi8
// will produce zeros for these positions. (Bitwise-xor of these
// masks with vsign will produce the opposite behavior.)
const __m128i mask_start
= _mm_sub_epi8(vramp, _mm_set1_epi8((char)skip_start));
const __m128i mask_end
= _mm_sub_epi8(vramp, _mm_set1_epi8((char)skip_end));
// Get the first 1-16 bytes into data0. If loading less than 16
// bytes, the bytes are loaded to the high bits of the vector and
// the least significant positions are filled with zeros.
const __m128i data0 = _mm_blendv_epi8(_mm_load_si128(aligned_buf),
_mm_setzero_si128(), mask_start);
aligned_buf++;
__m128i v2, v3;
#ifndef CRC_USE_GENERIC_FOR_SMALL_INPUTS
if (size <= 16) {
// Right-shift initial_crc by 1-16 bytes based on "size"
// and store the result in v1 (high bytes) and v0 (low bytes).
//
// NOTE: The highest 8 bytes of initial_crc are zeros so
// v1 will be filled with zeros if size >= 8. The highest
// 8 bytes of v1 will always become zeros.
//
// [ v1 ][ v0 ]
// [ initial_crc ] size == 1
// [ initial_crc ] size == 2
// [ initial_crc ] size == 15
// [ initial_crc ] size == 16 (all in v0)
const __m128i mask_low = _mm_add_epi8(
vramp, _mm_set1_epi8((char)(size - 16)));
MASK_LH(initial_crc, mask_low, *v0, *v1);
if (size2 <= 16) {
// There are 1-16 bytes of input and it is all
// in data0. Copy the input bytes to v3. If there
// are fewer than 16 bytes, the low bytes in v3
// will be filled with zeros. That is, the input
// bytes are stored to the same position as
// (part of) initial_crc is in v0.
MASK_L(data0, mask_end, v3);
} else {
// There are 2-16 bytes of input but not all bytes
// are in data0.
const __m128i data1 = _mm_load_si128(aligned_buf);
// Collect the 2-16 input bytes from data0 and data1
// to v2 and v3, and bitwise-xor them with the
// low bits of initial_crc in v0. Note that the
// the second xor is below this else-block as it
// is shared with the other branch.
MASK_H(data0, mask_end, v2);
MASK_L(data1, mask_end, v3);
*v0 = _mm_xor_si128(*v0, v2);
}
*v0 = _mm_xor_si128(*v0, v3);
*v1 = _mm_alignr_epi8(*v1, *v0, 8);
} else
#endif
{
// There is more than 16 bytes of input.
const __m128i data1 = _mm_load_si128(aligned_buf);
const __m128i *end = (const __m128i*)(
(const char *)aligned_buf - 16 + size2);
aligned_buf++;
MASK_LH(initial_crc, mask_start, *v0, *v1);
*v0 = _mm_xor_si128(*v0, data0);
*v1 = _mm_xor_si128(*v1, data1);
while (aligned_buf < end) {
*v1 = _mm_xor_si128(*v1, _mm_clmulepi64_si128(
*v0, vfold16, 0x00));
*v0 = _mm_xor_si128(*v1, _mm_clmulepi64_si128(
*v0, vfold16, 0x11));
*v1 = _mm_load_si128(aligned_buf++);
}
if (aligned_buf != end) {
MASK_H(*v0, mask_end, v2);
MASK_L(*v0, mask_end, *v0);
MASK_L(*v1, mask_end, v3);
*v1 = _mm_or_si128(v2, v3);
}
*v1 = _mm_xor_si128(*v1, _mm_clmulepi64_si128(
*v0, vfold16, 0x00));
*v0 = _mm_xor_si128(*v1, _mm_clmulepi64_si128(
*v0, vfold16, 0x11));
*v1 = _mm_srli_si128(*v0, 8);
}
}
/////////////////////
// x86 CLMUL CRC32 //
/////////////////////
/*
// These functions were used to generate the constants
// at the top of crc32_arch_optimized().
static uint64_t
calc_lo(uint64_t p, uint64_t a, int n)
{
uint64_t b = 0; int i;
for (i = 0; i < n; i++) {
b = b >> 1 | (a & 1) << (n - 1);
a = (a >> 1) ^ ((0 - (a & 1)) & p);
}
return b;
}
// same as ~crc(&a, sizeof(a), ~0)
static uint64_t
calc_hi(uint64_t p, uint64_t a, int n)
{
int i;
for (i = 0; i < n; i++)
a = (a >> 1) ^ ((0 - (a & 1)) & p);
return a;
}
*/
#ifdef BUILDING_CRC32_CLMUL
crc_attr_target
crc_attr_no_sanitize_address
static uint32_t
crc32_arch_optimized(const uint8_t *buf, size_t size, uint32_t crc)
{
#ifndef CRC_USE_GENERIC_FOR_SMALL_INPUTS
// The code assumes that there is at least one byte of input.
if (size == 0)
return crc;
#endif
// uint32_t poly = 0xedb88320;
const int64_t p = 0x1db710640; // p << 1
const int64_t mu = 0x1f7011641; // calc_lo(p, p, 32) << 1 | 1
const int64_t k5 = 0x163cd6124; // calc_hi(p, p, 32) << 1
const int64_t k4 = 0x0ccaa009e; // calc_hi(p, p, 64) << 1
const int64_t k3 = 0x1751997d0; // calc_hi(p, p, 128) << 1
const __m128i vfold4 = _mm_set_epi64x(mu, p);
const __m128i vfold8 = _mm_set_epi64x(0, k5);
const __m128i vfold16 = _mm_set_epi64x(k4, k3);
__m128i v0, v1, v2;
crc_simd_body(buf, size, &v0, &v1, vfold16,
_mm_cvtsi32_si128((int32_t)~crc));
v1 = _mm_xor_si128(
_mm_clmulepi64_si128(v0, vfold16, 0x10), v1); // xxx0
v2 = _mm_shuffle_epi32(v1, 0xe7); // 0xx0
v0 = _mm_slli_epi64(v1, 32); // [0]
v0 = _mm_clmulepi64_si128(v0, vfold8, 0x00);
v0 = _mm_xor_si128(v0, v2); // [1] [2]
v2 = _mm_clmulepi64_si128(v0, vfold4, 0x10);
v2 = _mm_clmulepi64_si128(v2, vfold4, 0x00);
v0 = _mm_xor_si128(v0, v2); // [2]
return ~(uint32_t)_mm_extract_epi32(v0, 2);
}
#endif // BUILDING_CRC32_CLMUL
/////////////////////
// x86 CLMUL CRC64 //
/////////////////////
/*
// These functions were used to generate the constants
// at the top of crc64_arch_optimized().
static uint64_t
calc_lo(uint64_t poly)
{
uint64_t a = poly;
uint64_t b = 0;
for (unsigned i = 0; i < 64; ++i) {
b = (b >> 1) | (a << 63);
a = (a >> 1) ^ (a & 1 ? poly : 0);
}
return b;
}
static uint64_t
calc_hi(uint64_t poly, uint64_t a)
{
for (unsigned i = 0; i < 64; ++i)
a = (a >> 1) ^ (a & 1 ? poly : 0);
return a;
}
*/
#ifdef BUILDING_CRC64_CLMUL
// MSVC (VS2015 - VS2022) produces bad 32-bit x86 code from the CLMUL CRC
// code when optimizations are enabled (release build). According to the bug
// report, the ebx register is corrupted and the calculated result is wrong.
// Trying to workaround the problem with "__asm mov ebx, ebx" didn't help.
// The following pragma works and performance is still good. x86-64 builds
// and CRC32 CLMUL aren't affected by this problem. The problem does not
// happen in crc_simd_body() either (which is shared with CRC32 CLMUL anyway).
//
// NOTE: Another pragma after crc64_arch_optimized() restores
// the optimizations. If the #if condition here is updated,
// the other one must be updated too.
#if defined(_MSC_VER) && !defined(__INTEL_COMPILER) && !defined(__clang__) \
&& defined(_M_IX86)
# pragma optimize("g", off)
#endif
crc_attr_target
crc_attr_no_sanitize_address
static uint64_t
crc64_arch_optimized(const uint8_t *buf, size_t size, uint64_t crc)
{
#ifndef CRC_USE_GENERIC_FOR_SMALL_INPUTS
// The code assumes that there is at least one byte of input.
if (size == 0)
return crc;
#endif
// const uint64_t poly = 0xc96c5795d7870f42; // CRC polynomial
const uint64_t p = 0x92d8af2baf0e1e85; // (poly << 1) | 1
const uint64_t mu = 0x9c3e466c172963d5; // (calc_lo(poly) << 1) | 1
const uint64_t k2 = 0xdabe95afc7875f40; // calc_hi(poly, 1)
const uint64_t k1 = 0xe05dd497ca393ae4; // calc_hi(poly, k2)
const __m128i vfold8 = _mm_set_epi64x((int64_t)p, (int64_t)mu);
const __m128i vfold16 = _mm_set_epi64x((int64_t)k2, (int64_t)k1);
__m128i v0, v1, v2;
#if defined(__i386__) || defined(_M_IX86)
crc_simd_body(buf, size, &v0, &v1, vfold16,
_mm_set_epi64x(0, (int64_t)~crc));
#else
// GCC and Clang would produce good code with _mm_set_epi64x
// but MSVC needs _mm_cvtsi64_si128 on x86-64.
crc_simd_body(buf, size, &v0, &v1, vfold16,
_mm_cvtsi64_si128((int64_t)~crc));
#endif
v1 = _mm_xor_si128(_mm_clmulepi64_si128(v0, vfold16, 0x10), v1);
v0 = _mm_clmulepi64_si128(v1, vfold8, 0x00);
v2 = _mm_clmulepi64_si128(v0, vfold8, 0x10);
v0 = _mm_xor_si128(_mm_xor_si128(v1, _mm_slli_si128(v0, 8)), v2);
#if defined(__i386__) || defined(_M_IX86)
return ~(((uint64_t)(uint32_t)_mm_extract_epi32(v0, 3) << 32) |
(uint64_t)(uint32_t)_mm_extract_epi32(v0, 2));
#else
return ~(uint64_t)_mm_extract_epi64(v0, 1);
#endif
}
#if defined(_MSC_VER) && !defined(__INTEL_COMPILER) && !defined(__clang__) \
&& defined(_M_IX86)
# pragma optimize("", on)
#endif
#endif // BUILDING_CRC64_CLMUL
// is_arch_extension_supported() must be inlined in this header file because
// the ifunc resolver function may not support calling a function in another
// translation unit. Depending on compiler-toolchain and flags, a call to
// a function defined in another translation unit could result in a
// reference to the PLT, which is unsafe to do in an ifunc resolver. The
// ifunc resolver runs very early when loading a shared library, so the PLT
// entries may not be setup at that time. Inlining this function duplicates
// the function body in crc32_resolve() and crc64_resolve(), but this is
// acceptable because the function results in very few instructions.
static inline bool
is_arch_extension_supported(void)
{
int success = 1;
uint32_t r[4]; // eax, ebx, ecx, edx
#if defined(_MSC_VER)
// This needs <intrin.h> with MSVC. ICC has it as a built-in
// on all platforms.
__cpuid(r, 1);
#elif defined(HAVE_CPUID_H)
// Compared to just using __asm__ to run CPUID, this also checks
// that CPUID is supported and saves and restores ebx as that is
// needed with GCC < 5 with position-independent code (PIC).
success = __get_cpuid(1, &r[0], &r[1], &r[2], &r[3]);
#else
// Just a fallback that shouldn't be needed.
__asm__("cpuid\n\t"
: "=a"(r[0]), "=b"(r[1]), "=c"(r[2]), "=d"(r[3])
: "a"(1), "c"(0));
#endif
// Returns true if these are supported:
// CLMUL (bit 1 in ecx)
// SSSE3 (bit 9 in ecx)
// SSE4.1 (bit 19 in ecx)
const uint32_t ecx_mask = (1 << 1) | (1 << 9) | (1 << 19);
return success && (r[2] & ecx_mask) == ecx_mask;
// Alternative methods that weren't used:
// - ICC's _may_i_use_cpu_feature: the other methods should work too.
// - GCC >= 6 / Clang / ICX __builtin_cpu_supports("pclmul")
//
// CPUID decding is needed with MSVC anyway and older GCC. This keeps
// the feature checks in the build system simpler too. The nice thing
// about __builtin_cpu_supports would be that it generates very short
// code as is it only reads a variable set at startup but a few bytes
// doesn't matter here.
}

View file

@ -1,5 +1,3 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file alone_decoder.c
@ -7,6 +5,9 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#include "alone_decoder.h"

View file

@ -1,5 +1,3 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file alone_decoder.h
@ -7,6 +5,9 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#ifndef LZMA_ALONE_DECODER_H

View file

@ -1,5 +1,3 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file alone_encoder.c
@ -7,6 +5,9 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#include "common.h"

View file

@ -1,5 +1,3 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file auto_decoder.c
@ -7,6 +5,9 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#include "stream_decoder.h"

View file

@ -1,5 +1,3 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file block_buffer_decoder.c
@ -7,6 +5,9 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#include "block_decoder.h"

View file

@ -1,5 +1,3 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file block_buffer_encoder.c
@ -7,6 +5,9 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#include "block_buffer_encoder.h"

View file

@ -1,5 +1,3 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file block_buffer_encoder.h
@ -7,6 +5,9 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#ifndef LZMA_BLOCK_BUFFER_ENCODER_H

View file

@ -1,5 +1,3 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file block_decoder.c
@ -7,6 +5,9 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#include "block_decoder.h"

View file

@ -1,5 +1,3 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file block_decoder.h
@ -7,6 +5,9 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#ifndef LZMA_BLOCK_DECODER_H

View file

@ -1,5 +1,3 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file block_encoder.c
@ -7,6 +5,9 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#include "block_encoder.h"

View file

@ -1,5 +1,3 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file block_encoder.h
@ -7,6 +5,9 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#ifndef LZMA_BLOCK_ENCODER_H

View file

@ -1,5 +1,3 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file block_header_decoder.c
@ -7,6 +5,9 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#include "common.h"

View file

@ -1,5 +1,3 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file block_header_encoder.c
@ -7,6 +5,9 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#include "common.h"

View file

@ -1,5 +1,3 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file block_util.c
@ -7,6 +5,9 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#include "common.h"

View file

@ -1,5 +1,3 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file common.c
@ -7,6 +5,9 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#include "common.h"

View file

@ -1,5 +1,3 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file common.h
@ -7,6 +5,9 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#ifndef LZMA_COMMON_H

View file

@ -1,5 +1,3 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file easy_buffer_encoder.c
@ -7,6 +5,9 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#include "easy_preset.h"

View file

@ -1,5 +1,3 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file easy_decoder_memusage.c
@ -7,6 +5,9 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#include "easy_preset.h"

View file

@ -1,5 +1,3 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file easy_encoder.c
@ -7,6 +5,9 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#include "easy_preset.h"

View file

@ -1,5 +1,3 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file easy_encoder_memusage.c
@ -7,6 +5,9 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#include "easy_preset.h"

View file

@ -1,5 +1,3 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file easy_preset.c
@ -7,6 +5,9 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#include "easy_preset.h"

View file

@ -1,5 +1,3 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file easy_preset.h
@ -7,6 +5,9 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#include "common.h"

View file

@ -1,5 +1,3 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file file_info.c
@ -7,6 +5,9 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#include "index_decoder.h"

View file

@ -1,5 +1,3 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file filter_buffer_decoder.c
@ -7,6 +5,9 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#include "filter_decoder.h"

View file

@ -1,5 +1,3 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file filter_buffer_encoder.c
@ -7,6 +5,9 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#include "filter_encoder.h"

View file

@ -1,5 +1,3 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file filter_common.c
@ -7,6 +5,9 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#include "filter_common.h"
@ -121,15 +122,6 @@ static const struct {
.changes_size = false,
},
#endif
#if defined(HAVE_ENCODER_RISCV) || defined(HAVE_DECODER_RISCV)
{
.id = LZMA_FILTER_RISCV,
.options_size = sizeof(lzma_options_bcj),
.non_last_ok = true,
.last_ok = false,
.changes_size = false,
},
#endif
#if defined(HAVE_ENCODER_DELTA) || defined(HAVE_DECODER_DELTA)
{
.id = LZMA_FILTER_DELTA,

View file

@ -1,5 +1,3 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file filter_common.h
@ -7,6 +5,9 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#ifndef LZMA_FILTER_COMMON_H

View file

@ -1,5 +1,3 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file filter_decoder.c
@ -7,6 +5,9 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#include "filter_decoder.h"
@ -120,14 +121,6 @@ static const lzma_filter_decoder decoders[] = {
.props_decode = &lzma_simple_props_decode,
},
#endif
#ifdef HAVE_DECODER_RISCV
{
.id = LZMA_FILTER_RISCV,
.init = &lzma_simple_riscv_decoder_init,
.memusage = NULL,
.props_decode = &lzma_simple_props_decode,
},
#endif
#ifdef HAVE_DECODER_DELTA
{
.id = LZMA_FILTER_DELTA,

View file

@ -1,5 +1,3 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file filter_decoder.h
@ -7,6 +5,9 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#ifndef LZMA_FILTER_DECODER_H

View file

@ -1,5 +1,3 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file filter_decoder.c
@ -7,6 +5,9 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#include "filter_encoder.h"
@ -32,8 +33,7 @@ typedef struct {
/// Calculates the recommended Uncompressed Size for .xz Blocks to
/// which the input data can be split to make multithreaded
/// encoding possible. If this is NULL, it is assumed that
/// the encoder is fast enough with single thread. If the options
/// are invalid, UINT64_MAX is returned.
/// the encoder is fast enough with single thread.
uint64_t (*block_size)(const void *options);
/// Tells the size of the Filter Properties field. If options are
@ -158,16 +158,6 @@ static const lzma_filter_encoder encoders[] = {
.props_encode = &lzma_simple_props_encode,
},
#endif
#ifdef HAVE_ENCODER_RISCV
{
.id = LZMA_FILTER_RISCV,
.init = &lzma_simple_riscv_encoder_init,
.memusage = NULL,
.block_size = NULL,
.props_size_get = &lzma_simple_props_size,
.props_encode = &lzma_simple_props_encode,
},
#endif
#ifdef HAVE_ENCODER_DELTA
{
.id = LZMA_FILTER_DELTA,
@ -229,17 +219,17 @@ lzma_filters_update(lzma_stream *strm, const lzma_filter *filters)
extern lzma_ret
lzma_raw_encoder_init(lzma_next_coder *next, const lzma_allocator *allocator,
const lzma_filter *filters)
const lzma_filter *options)
{
return lzma_raw_coder_init(next, allocator,
filters, (lzma_filter_find)(&encoder_find), true);
options, (lzma_filter_find)(&encoder_find), true);
}
extern LZMA_API(lzma_ret)
lzma_raw_encoder(lzma_stream *strm, const lzma_filter *filters)
lzma_raw_encoder(lzma_stream *strm, const lzma_filter *options)
{
lzma_next_strm_init(lzma_raw_coder_init, strm, filters,
lzma_next_strm_init(lzma_raw_coder_init, strm, options,
(lzma_filter_find)(&encoder_find), true);
strm->internal->supported_actions[LZMA_RUN] = true;
@ -258,29 +248,26 @@ lzma_raw_encoder_memusage(const lzma_filter *filters)
}
extern LZMA_API(uint64_t)
extern uint64_t
lzma_mt_block_size(const lzma_filter *filters)
{
if (filters == NULL)
return UINT64_MAX;
uint64_t max = 0;
for (size_t i = 0; filters[i].id != LZMA_VLI_UNKNOWN; ++i) {
const lzma_filter_encoder *const fe
= encoder_find(filters[i].id);
if (fe == NULL)
return UINT64_MAX;
if (fe->block_size != NULL) {
const uint64_t size
= fe->block_size(filters[i].options);
if (size == 0)
return 0;
if (size > max)
max = size;
}
}
return max == 0 ? UINT64_MAX : max;
return max;
}

View file

@ -1,12 +1,13 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file filter_encoder.h
/// \file filter_encoder.c
/// \brief Filter ID mapping to filter-specific functions
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#ifndef LZMA_FILTER_ENCODER_H
@ -15,6 +16,10 @@
#include "common.h"
// FIXME: Might become a part of the public API.
extern uint64_t lzma_mt_block_size(const lzma_filter *filters);
extern lzma_ret lzma_raw_encoder_init(
lzma_next_coder *next, const lzma_allocator *allocator,
const lzma_filter *filters);

View file

@ -1,5 +1,3 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file filter_flags_decoder.c
@ -7,6 +5,9 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#include "filter_decoder.h"

View file

@ -1,5 +1,3 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file filter_flags_encoder.c
@ -7,6 +5,9 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#include "filter_encoder.h"

View file

@ -1,5 +1,3 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file hardware_cputhreads.c
@ -7,6 +5,9 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#include "common.h"

View file

@ -1,5 +1,3 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file hardware_physmem.c
@ -7,6 +5,9 @@
//
// Author: Jonathan Nieder
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#include "common.h"

Some files were not shown because too many files have changed in this diff Show more