This change implements a new bytecode compiler, based on a
transformation of the parse tree to an abstract syntax defined in
Parser/Python.asdl.
The compiler implementation is not complete, but it is in stable
enough shape to run the entire test suite excepting two disabled
tests.
- SF Bug #772896, unknown encoding results in MemoryError, which is not helpful
I will only backport the segfault fix. I'll let Anthony decide if he wants
the other changes backported. I will do the backport if asked.
Fix over-aggressive PyErr_Clear(). The same code fragment appears in
various guises in list.extend(), map(), filter(), zip(), and internally
in PySequence_Tuple().
- Handle both frozenset() and frozenset([]).
- Do not use singleton for frozenset subclasses.
- Finalize the singleton.
- Add test cases.
* Factor-out set_update_internal() from set_update(). Simplifies the
code for several internal callers.
* Factor constant expressions out of loop in set_merge_internal().
* Minor comment touch-ups.
[ 1180995 ] binary formats for marshalling floats
Adds 2 new type codes for marshal (binary floats and binary complexes), a
new marshal version (2), updates MAGIC and fiddles the de-serializing of
code objects to be less likely to clobber the real reason for failing if
it fails.
[ 1181301 ] make float packing copy bytes when they can
which hasn't been reviewed, despite numerous threats to check it in
anyway if noone reviews it. Please read the diff on the checkin list,
at least!
The basic idea is to examine the bytes of some 'probe values' to see if
the current platform is a IEEE 754-ish platform, and if so
_PyFloat_{Pack,Unpack}{4,8} just copy bytes around.
The rest is hair for testing, and tests.
A problem regarding importing symlinked modules was recently reported on the
Cygwin mailing list:
http://cygwin.com/ml/cygwin/2005-04/msg00257.html
The following test case demonstrates the problem:
$ ls -l
total 1
lrwxrwxrwx 1 jt None 6 Apr 23 13:32 bar.py -> foo.py
-rw-r--r-- 1 jt None 24 Apr 18 20:13 foo.py
$ python -c 'import bar'
Traceback (most recent call last):
File "<string>", line 1, in ?
ImportError: No module named bar
Since Cygwin's case_ok() uses a modified version of the Windows's version, the
symlinked bar module actually resolves to file foo.py instead of bar.py. This
obviously causes the matching code to fail (regardless of case).
The patch fixes this problem by making Cygwin use the Mac OS X case_ok()
instead of a modified Window's version.
If we exit via the break here, we need to set ff_last_lineno or
FUTURE_POSSIBLE() will remain true. The bug affected statements
containing a variety of expressions, but not all expressions. It has
been present since Python 2.2.
Improve signal handling, especially when using threads, by forcing an early
re-execution of PyEval_EvalFrame() "periodic" code when things_to_do is not
cleared by Py_MakePendingCalls().
M Misc/NEWS
M Python/ceval.c
PyGILState_Ensure(): The fix in 2.4a3 for bug 1010677 reintroduced thread
shutdown race bug 225673. Repaired by (once again) ensuring the GIL is
held whenever deleting a thread state.
Alas, there's no useful test case for this shy bug. Four years ago, only
Guido could provoke it, on his box, and today only Armin can provoke it
on his box. I've never been able to provoke it (but not for lack of
trying!).
This is a critical fix for 2.3.5 too, since the fix for 1010677 got
backported there already and so also reintroduced 225673. I don't intend to
backport this fix. For whoever (if anyone) does, there are other thread
fixes in 2.4 that need backporting too, and I bet they need to happen first
for this patch to apply cleanly.
(Contributed by Bob Ippolito.)
This patch trims down the Python core on Darwin by making it
independent of CoreFoundation and CoreServices. It does this by:
Changed linker flags in configure/configure.in
Removed the unused PyMac_GetAppletScriptFile
Moved the implementation of PyMac_StrError to the MacOS module
Moved the implementation of PyMac_GetFullPathname to the
Carbon.File module
* Use simpler, faster two pass algorithm for markblocks().
* Free the blocks variable if not NULL and exiting without change.
* Verify that the rest of the compiler has not set an exception.
* Make the test for tuple of constants less restrictive.
* Embellish the comment for chained conditional jumps.
Peepholer could be fooled into misidentifying a tuple_of_constants.
Added code to count consecutive occurrences of LOAD_CONST.
Use the count to weed out the misidentified cases.
Added a unittest.
thread's id can't get duplicated, because (of course!) the current thread
is still running. The code should work either way, but reverting the
gratuitous change should make backporting easier, and gets the bad
reasoning out of 2.35's new comments.
can fail, check its return value, and die if it does fail.
_PyGILState_Init(): Assert that the thread doesn't already have an
association for autoTLSkey. If it does, PyThread_set_key_value() will
ignore the attempt to (re)set the association, which the code clearly
doesn't want.
code.
PyThread_set_key_value(): It's clear that this code assumes the passed-in
value isn't NULL, so document that it must not be, and assert that it
isn't. It remains unclear whether existing callers want the odd semantics
actually implemented by this function.
High level error message was stomping useful detailed messages from lower
level routines.
The new approach is to augment string error messages returned by the low
level routines. The provides both high and low level information. If
the exception value is not a string, no changes are made.
To see the improved messages in action, type:
import random
class R(random): pass
class B(bool): pass
Allows the lineno fixup code to remain simple and not have to deal with
multibyte codings.
* Add an assertion to that effect.
* Remove the XXX comment on the subject.
happen in 2.3, but nobody noticed it still was getting generated (the
warning was disabled by default). OverflowWarning and
PyExc_OverflowWarning should be removed for 2.5, and left notes all over
saying so.
* Perform the code length check earlier.
* Eliminate the extra PyMem_Free() upon hitting an EXTENDED_ARG.
* Assert that the NOP count used in jump retargeting matches the NOPs
eliminated in the final step.
* Add an XXX note to indicate that more work is being to done to
handle linenotab with intervals > 255.
* Make a pass to eliminate NOPs. Produce code that is more readable,
more compact, and a tiny bit faster. Makes the peepholer more flexible
in the scope of allowable transformations.
* With Guido's okay, bumped up the magic number so that this patch gets
widely exercised before the alpha goes out.
files and not re-optimized upon import. Saves a bit of startup time while
still remaining decoupled from the rest of the compiler.
As a side benefit, handcoded bytecode is not run through the optimizer
when new code objects are created. Hopefully, a handcoder has already
created exactly what they want to have run.
(Idea suggested by Armin Rigo and Michael Hudson. Initially avoided
because of worries about compiler coupling; however, only the nexus
point needed to be moved so there won't be a conflict when the AST
branch is loaded.)
[ 1009560 ] Fix @decorator evaluation order
From the description:
Changes in this patch:
- Change Grammar/Grammar to require
newlines between adjacent decorators.
- Fix order of evaluation of decorators
in the C (compile.c) and python
(Lib/compiler/pycodegen.py) compilers
- Add better order of evaluation check
to test_decorators.py (test_eval_order)
- Update the decorator documentation in
the reference manual (improve description
of evaluation order and update syntax
description)
and the comment:
Used Brett's evaluation order (see
http://mail.python.org/pipermail/python-dev/2004-August/047835.html)
(I'm checking this in for Anthony who was having problems getting SF to
talk to him)
because GNU/k*BSD uses gnu pth to provide pthreads, but will also happen on any
system that does the same.
python fails to build because it doesn't detect gnu pth in pthread
emulation. See C comments in patch for details.
patch taken from http://bugs.debian.org/264315
[ 1005248 ] new.code() not cleanly checking its arguments
using the result of new.code() can still destroy the sun, but merely
calling the function shouldn't any more.
I also rewrote the existing tests of new.code() to use vastly less
un-bogus arguments, and added tests for the previous insane behaviours.
hack: it would resize *interned* strings in-place! This occurred because
their reference counts do not have their expected value -- stringobject.c
hacks them. Mea culpa.
interning were not clear here -- a subclass could be mutable, for
example -- and had bugs. Explicitly interning a subclass of string
via intern() will raise a TypeError. Internal operations that attempt
to intern a string subclass will have no effect.
Added a few tests to test_builtin that includes the old buggy code and
verifies that calls like PyObject_SetAttr() don't fail. Perhaps these
tests should have gone in test_string.
[ 991812 ] PyArg_ParseTuple can miss errors with warnings as exceptions
as suggested in the report.
This is definitely a 2.3 candidate (as are most of the checkins I've
made in the last month...)
have differing refcount semantics. If anyone sees a prettier way to
acheive the same ends, then please go for it.
I think this is the first time I've ever used Py_XINCREF.
* Fixes an incorrect variable in a PyDict_CheckExact.
* Allow general mapping locals arguments for the execfile() function
and exec statement.
* Add tests.
PyImport_ReloadModule(): restore the module to sys.modules in error cases.
load_package(): semantic-neutral refactoring from an earlier stab at
this patch; giving it a common error exit made the code
easier to follow, so retaining that part.
_RemoveModule(): new little utility to delete a key from sys.modules.
__oct__, and __hex__. Raise TypeError if an invalid type is
returned. Note that PyNumber_Int and PyNumber_Long can still
return ints or longs. Fixes SF bug #966618.
The preceding case statement was missing a terminating "break" stmt,
so fell into the new code by mistake. This caused uncaught out-of-bounds
accesses to the "names" tuple, leading to a variety of insane behaviors.
expected entrypoint. The unlinking will crash the application if the
module contained ObjC code. The price of this is small: a little wasted
memory, and only in a case than isn't expected to occur often.
[ 984722 ] Py_BuildValue loses reference counts on error
I'm ever-so-slightly uneasy at the amount of work this can do with an
exception pending, but I don't think that this can result in anything
more serious than a strange error message.
