bpo-38787: C API for module state access from extension methods (PEP 573) (GH-19936)

Module C state is now accessible from C-defined heap type methods (PEP 573).
Patch by Marcel Plch and Petr Viktorin.

Co-authored-by: Marcel Plch <mplch@redhat.com>
Co-authored-by: Victor Stinner <vstinner@python.org>
This commit is contained in:
Petr Viktorin 2020-05-07 15:39:59 +02:00 committed by GitHub
parent 4638c64295
commit e1becf46b4
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
19 changed files with 797 additions and 51 deletions

View file

@ -147,23 +147,56 @@ Implementing functions and methods
value of the function as exposed in Python. The function must return a new
reference.
The function signature is::
PyObject *PyCFunction(PyObject *self,
PyObject *const *args);
.. c:type:: PyCFunctionWithKeywords
Type of the functions used to implement Python callables in C
with signature :const:`METH_VARARGS | METH_KEYWORDS`.
The function signature is::
PyObject *PyCFunctionWithKeywords(PyObject *self,
PyObject *const *args,
PyObject *kwargs);
.. c:type:: _PyCFunctionFast
Type of the functions used to implement Python callables in C
with signature :const:`METH_FASTCALL`.
The function signature is::
PyObject *_PyCFunctionFast(PyObject *self,
PyObject *const *args,
Py_ssize_t nargs);
.. c:type:: _PyCFunctionFastWithKeywords
Type of the functions used to implement Python callables in C
with signature :const:`METH_FASTCALL | METH_KEYWORDS`.
The function signature is::
PyObject *_PyCFunctionFastWithKeywords(PyObject *self,
PyObject *const *args,
Py_ssize_t nargs,
PyObject *kwnames);
.. c:type:: PyCMethod
Type of the functions used to implement Python callables in C
with signature :const:`METH_METHOD | METH_FASTCALL | METH_KEYWORDS`.
The function signature is::
PyObject *PyCMethod(PyObject *self,
PyTypeObject *defining_class,
PyObject *const *args,
Py_ssize_t nargs,
PyObject *kwnames)
.. versionadded:: 3.9
.. c:type:: PyMethodDef
@ -197,9 +230,7 @@ The :attr:`ml_flags` field is a bitfield which can include the following flags.
The individual flags indicate either a calling convention or a binding
convention.
There are four basic calling conventions for positional arguments
and two of them can be combined with :const:`METH_KEYWORDS` to support
also keyword arguments. So there are a total of 6 calling conventions:
There are these calling conventions:
.. data:: METH_VARARGS
@ -250,6 +281,19 @@ also keyword arguments. So there are a total of 6 calling conventions:
.. versionadded:: 3.7
.. data:: METH_METHOD | METH_FASTCALL | METH_KEYWORDS
Extension of :const:`METH_FASTCALL | METH_KEYWORDS` supporting the *defining
class*, that is, the class that contains the method in question.
The defining class might be a superclass of ``Py_TYPE(self)``.
The method needs to be of type :c:type:`PyCMethod`, the same as for
``METH_FASTCALL | METH_KEYWORDS`` with ``defining_class`` argument added after
``self``.
.. versionadded:: 3.9
.. data:: METH_NOARGS
Methods without parameters don't need to check whether arguments are given if

View file

@ -109,6 +109,30 @@ Type Objects
.. versionadded:: 3.4
.. c:function:: PyObject* PyType_GetModule(PyTypeObject *type)
Return the module object associated with the given type when the type was
created using :c:func:`PyType_FromModuleAndSpec`.
If no module is associated with the given type, sets :py:class:`TypeError`
and returns ``NULL``.
.. versionadded:: 3.9
.. c:function:: void* PyType_GetModuleState(PyTypeObject *type)
Return the state of the module object associated with the given type.
This is a shortcut for calling :c:func:`PyModule_GetState()` on the result
of :c:func:`PyType_GetModule`.
If no module is associated with the given type, sets :py:class:`TypeError`
and returns ``NULL``.
If the *type* has an associated module but its state is ``NULL``,
returns ``NULL`` without setting an exception.
.. versionadded:: 3.9
Creating Heap-Allocated Types
.............................
@ -116,7 +140,7 @@ Creating Heap-Allocated Types
The following functions and structs are used to create
:ref:`heap types <heap-types>`.
.. c:function:: PyObject* PyType_FromSpecWithBases(PyType_Spec *spec, PyObject *bases)
.. c:function:: PyObject* PyType_FromModuleAndSpec(PyObject *module, PyType_Spec *spec, PyObject *bases)
Creates and returns a heap type object from the *spec*
(:const:`Py_TPFLAGS_HEAPTYPE`).
@ -127,8 +151,18 @@ The following functions and structs are used to create
If *bases* is ``NULL``, the *Py_tp_base* slot is used instead.
If that also is ``NULL``, the new type derives from :class:`object`.
The *module* must be a module object or ``NULL``.
If not ``NULL``, the module is associated with the new type and can later be
retreived with :c:func:`PyType_GetModule`.
This function calls :c:func:`PyType_Ready` on the new type.
.. versionadded:: 3.9
.. c:function:: PyObject* PyType_FromSpecWithBases(PyType_Spec *spec, PyObject *bases)
Equivalent to ``PyType_FromModuleAndSpec(NULL, spec, bases)``.
.. versionadded:: 3.3
.. c:function:: PyObject* PyType_FromSpec(PyType_Spec *spec)

View file

@ -0,0 +1,32 @@
#ifndef Py_CPYTHON_METHODOBJECT_H
# error "this header file must not be included directly"
#endif
PyAPI_DATA(PyTypeObject) PyCMethod_Type;
/* Macros for direct access to these values. Type checks are *not*
done, so use with care. */
#define PyCFunction_GET_FUNCTION(func) \
(((PyCFunctionObject *)func) -> m_ml -> ml_meth)
#define PyCFunction_GET_SELF(func) \
(((PyCFunctionObject *)func) -> m_ml -> ml_flags & METH_STATIC ? \
NULL : ((PyCFunctionObject *)func) -> m_self)
#define PyCFunction_GET_FLAGS(func) \
(((PyCFunctionObject *)func) -> m_ml -> ml_flags)
#define PyCFunction_GET_CLASS(func) \
(((PyCFunctionObject *)func) -> m_ml -> ml_flags & METH_METHOD ? \
((PyCMethodObject *)func) -> mm_class : NULL)
typedef struct {
PyObject_HEAD
PyMethodDef *m_ml; /* Description of the C function to call */
PyObject *m_self; /* Passed as 'self' arg to the C func, can be NULL */
PyObject *m_module; /* The __module__ attribute, can be anything */
PyObject *m_weakreflist; /* List of weak references */
vectorcallfunc vectorcall;
} PyCFunctionObject;
typedef struct {
PyCFunctionObject func;
PyTypeObject *mm_class; /* Class that defines this method */
} PyCMethodObject;

View file

@ -289,6 +289,7 @@ typedef struct _heaptypeobject {
PyBufferProcs as_buffer;
PyObject *ht_name, *ht_slots, *ht_qualname;
struct _dictkeysobject *ht_cached_keys;
PyObject *ht_module;
/* here are optional user slots, followed by the members. */
} PyHeapTypeObject;

View file

@ -13,7 +13,7 @@ extern "C" {
PyAPI_DATA(PyTypeObject) PyCFunction_Type;
#define PyCFunction_Check(op) Py_IS_TYPE(op, &PyCFunction_Type)
#define PyCFunction_Check(op) (Py_IS_TYPE(op, &PyCFunction_Type) || (PyType_IsSubtype(Py_TYPE(op), &PyCFunction_Type)))
typedef PyObject *(*PyCFunction)(PyObject *, PyObject *);
typedef PyObject *(*_PyCFunctionFast) (PyObject *, PyObject *const *, Py_ssize_t);
@ -22,21 +22,13 @@ typedef PyObject *(*PyCFunctionWithKeywords)(PyObject *, PyObject *,
typedef PyObject *(*_PyCFunctionFastWithKeywords) (PyObject *,
PyObject *const *, Py_ssize_t,
PyObject *);
typedef PyObject *(*PyCMethod)(PyObject *, PyTypeObject *, PyObject *const *,
size_t, PyObject *);
PyAPI_FUNC(PyCFunction) PyCFunction_GetFunction(PyObject *);
PyAPI_FUNC(PyObject *) PyCFunction_GetSelf(PyObject *);
PyAPI_FUNC(int) PyCFunction_GetFlags(PyObject *);
/* Macros for direct access to these values. Type checks are *not*
done, so use with care. */
#ifndef Py_LIMITED_API
#define PyCFunction_GET_FUNCTION(func) \
(((PyCFunctionObject *)func) -> m_ml -> ml_meth)
#define PyCFunction_GET_SELF(func) \
(((PyCFunctionObject *)func) -> m_ml -> ml_flags & METH_STATIC ? \
NULL : ((PyCFunctionObject *)func) -> m_self)
#define PyCFunction_GET_FLAGS(func) \
(((PyCFunctionObject *)func) -> m_ml -> ml_flags)
#endif
Py_DEPRECATED(3.9) PyAPI_FUNC(PyObject *) PyCFunction_Call(PyObject *, PyObject *, PyObject *);
struct PyMethodDef {
@ -52,6 +44,13 @@ typedef struct PyMethodDef PyMethodDef;
PyAPI_FUNC(PyObject *) PyCFunction_NewEx(PyMethodDef *, PyObject *,
PyObject *);
#if !defined(Py_LIMITED_API) || Py_LIMITED_API+0 >= 0x03090000
#define PyCFunction_NewEx(ML, SELF, MOD) PyCMethod_New((ML), (SELF), (MOD), NULL)
PyAPI_FUNC(PyObject *) PyCMethod_New(PyMethodDef *, PyObject *,
PyObject *, PyTypeObject *);
#endif
/* Flag passed to newmethodobject */
/* #define METH_OLDARGS 0x0000 -- unsupported now */
#define METH_VARARGS 0x0001
@ -84,15 +83,24 @@ PyAPI_FUNC(PyObject *) PyCFunction_NewEx(PyMethodDef *, PyObject *,
#define METH_STACKLESS 0x0000
#endif
/* METH_METHOD means the function stores an
* additional reference to the class that defines it;
* both self and class are passed to it.
* It uses PyCMethodObject instead of PyCFunctionObject.
* May not be combined with METH_NOARGS, METH_O, METH_CLASS or METH_STATIC.
*/
#if !defined(Py_LIMITED_API) || Py_LIMITED_API+0 >= 0x03090000
#define METH_METHOD 0x0200
#endif
#ifndef Py_LIMITED_API
typedef struct {
PyObject_HEAD
PyMethodDef *m_ml; /* Description of the C function to call */
PyObject *m_self; /* Passed as 'self' arg to the C func, can be NULL */
PyObject *m_module; /* The __module__ attribute, can be anything */
PyObject *m_weakreflist; /* List of weak references */
vectorcallfunc vectorcall;
} PyCFunctionObject;
#define Py_CPYTHON_METHODOBJECT_H
#include "cpython/methodobject.h"
#undef Py_CPYTHON_METHODOBJECT_H
#endif
#ifdef __cplusplus

View file

@ -213,6 +213,11 @@ PyAPI_FUNC(PyObject*) PyType_FromSpecWithBases(PyType_Spec*, PyObject*);
#if !defined(Py_LIMITED_API) || Py_LIMITED_API+0 >= 0x03040000
PyAPI_FUNC(void*) PyType_GetSlot(PyTypeObject*, int);
#endif
#if !defined(Py_LIMITED_API) || Py_LIMITED_API+0 >= 0x03090000
PyAPI_FUNC(PyObject*) PyType_FromModuleAndSpec(PyObject *, PyType_Spec *, PyObject *);
PyAPI_FUNC(PyObject *) PyType_GetModule(struct _typeobject *);
PyAPI_FUNC(void *) PyType_GetModuleState(struct _typeobject *);
#endif
/* Generic type check */
PyAPI_FUNC(int) PyType_IsSubtype(PyTypeObject *, PyTypeObject *);

View file

@ -13,6 +13,8 @@
import time
import unittest
import weakref
import importlib.machinery
import importlib.util
from test import support
from test.support import MISSING_C_DOCSTRINGS
from test.support.script_helper import assert_python_failure, assert_python_ok
@ -774,5 +776,76 @@ class PyMemDefaultTests(PyMemDebugTests):
PYTHONMALLOC = ''
class Test_ModuleStateAccess(unittest.TestCase):
"""Test access to module start (PEP 573)"""
# The C part of the tests lives in _testmultiphase, in a module called
# _testmultiphase_meth_state_access.
# This module has multi-phase initialization, unlike _testcapi.
def setUp(self):
fullname = '_testmultiphase_meth_state_access' # XXX
origin = importlib.util.find_spec('_testmultiphase').origin
loader = importlib.machinery.ExtensionFileLoader(fullname, origin)
spec = importlib.util.spec_from_loader(fullname, loader)
module = importlib.util.module_from_spec(spec)
loader.exec_module(module)
self.module = module
def test_subclass_get_module(self):
"""PyType_GetModule for defining_class"""
class StateAccessType_Subclass(self.module.StateAccessType):
pass
instance = StateAccessType_Subclass()
self.assertIs(instance.get_defining_module(), self.module)
def test_subclass_get_module_with_super(self):
class StateAccessType_Subclass(self.module.StateAccessType):
def get_defining_module(self):
return super().get_defining_module()
instance = StateAccessType_Subclass()
self.assertIs(instance.get_defining_module(), self.module)
def test_state_access(self):
"""Checks methods defined with and without argument clinic
This tests a no-arg method (get_count) and a method with
both a positional and keyword argument.
"""
a = self.module.StateAccessType()
b = self.module.StateAccessType()
methods = {
'clinic': a.increment_count_clinic,
'noclinic': a.increment_count_noclinic,
}
for name, increment_count in methods.items():
with self.subTest(name):
self.assertEqual(a.get_count(), b.get_count())
self.assertEqual(a.get_count(), 0)
increment_count()
self.assertEqual(a.get_count(), b.get_count())
self.assertEqual(a.get_count(), 1)
increment_count(3)
self.assertEqual(a.get_count(), b.get_count())
self.assertEqual(a.get_count(), 4)
increment_count(-2, twice=True)
self.assertEqual(a.get_count(), b.get_count())
self.assertEqual(a.get_count(), 0)
with self.assertRaises(TypeError):
increment_count(thrice=3)
with self.assertRaises(TypeError):
increment_count(1, 2, 3)
if __name__ == "__main__":
unittest.main()

View file

@ -1322,7 +1322,7 @@ def delx(self): del self.__x
'3P' # PyMappingMethods
'10P' # PySequenceMethods
'2P' # PyBufferProcs
'4P')
'5P')
class newstyleclass(object): pass
# Separate block for PyDictKeysObject with 8 keys and 5 entries
check(newstyleclass, s + calcsize("2nP2n0P") + 8 + 5*calcsize("n2P"))

View file

@ -1104,6 +1104,7 @@ PYTHON_HEADERS= \
$(srcdir)/Include/cpython/initconfig.h \
$(srcdir)/Include/cpython/interpreteridobject.h \
$(srcdir)/Include/cpython/listobject.h \
$(srcdir)/Include/cpython/methodobject.h \
$(srcdir)/Include/cpython/object.h \
$(srcdir)/Include/cpython/objimpl.h \
$(srcdir)/Include/cpython/pyerrors.h \

View file

@ -0,0 +1,2 @@
Module C state is now accessible from C-defined heap type methods (:pep:`573`).
Patch by Marcel Plch and Petr Viktorin.

View file

@ -4,6 +4,19 @@
#include "Python.h"
/* State for testing module state access from methods */
typedef struct {
int counter;
} meth_state;
/*[clinic input]
module _testmultiphase
class _testmultiphase.StateAccessType "StateAccessTypeObject *" "!StateAccessType"
[clinic start generated code]*/
/*[clinic end generated code: output=da39a3ee5e6b4b0d input=bab9f2fe3bd312ff]*/
/* Example objects */
typedef struct {
PyObject_HEAD
@ -14,6 +27,10 @@ typedef struct {
PyObject *integer;
} testmultiphase_state;
typedef struct {
PyObject_HEAD
} StateAccessTypeObject;
/* Example methods */
static int
@ -42,6 +59,7 @@ Example_demo(ExampleObject *self, PyObject *args)
Py_RETURN_NONE;
}
#include "clinic/_testmultiphase.c.h"
static PyMethodDef Example_methods[] = {
{"demo", (PyCFunction)Example_demo, METH_VARARGS,
@ -102,6 +120,150 @@ static PyType_Spec Example_Type_spec = {
Example_Type_slots
};
/*[clinic input]
_testmultiphase.StateAccessType.get_defining_module
cls: defining_class
Return the module of the defining class.
[clinic start generated code]*/
static PyObject *
_testmultiphase_StateAccessType_get_defining_module_impl(StateAccessTypeObject *self,
PyTypeObject *cls)
/*[clinic end generated code: output=ba2a14284a5d0921 input=946149f91cf72c0d]*/
{
PyObject *retval;
retval = PyType_GetModule(cls);
if (retval == NULL) {
return NULL;
}
Py_INCREF(retval);
return retval;
}
/*[clinic input]
_testmultiphase.StateAccessType.increment_count_clinic
cls: defining_class
/
n: int = 1
*
twice: bool = False
Add 'n' from the module-state counter.
Pass 'twice' to double that amount.
This tests Argument Clinic support for defining_class.
[clinic start generated code]*/
static PyObject *
_testmultiphase_StateAccessType_increment_count_clinic_impl(StateAccessTypeObject *self,
PyTypeObject *cls,
int n, int twice)
/*[clinic end generated code: output=3b34f86bc5473204 input=551d482e1fe0b8f5]*/
{
meth_state *m_state = PyType_GetModuleState(cls);
if (twice) {
n *= 2;
}
m_state->counter += n;
Py_RETURN_NONE;
}
PyDoc_STRVAR(_StateAccessType_decrement_count__doc__,
"decrement_count($self, /, n=1, *, twice=None)\n"
"--\n"
"\n"
"Add 'n' from the module-state counter.\n"
"Pass 'twice' to double that amount.\n"
"(This is to test both positional and keyword arguments.");
// Intentionally does not use Argument Clinic
static PyObject *
_StateAccessType_increment_count_noclinic(StateAccessTypeObject *self,
PyTypeObject *defining_class,
PyObject *const *args,
Py_ssize_t nargs,
PyObject *kwnames)
{
if (!_PyArg_CheckPositional("StateAccessTypeObject.decrement_count", nargs, 0, 1)) {
return NULL;
}
long n = 1;
if (nargs) {
n = PyLong_AsLong(args[0]);
if (PyErr_Occurred()) {
return NULL;
}
}
if (kwnames && PyTuple_Check(kwnames)) {
if (PyTuple_GET_SIZE(kwnames) > 1 ||
PyUnicode_CompareWithASCIIString(
PyTuple_GET_ITEM(kwnames, 0),
"twice"
)) {
PyErr_SetString(
PyExc_TypeError,
"decrement_count only takes 'twice' keyword argument"
);
return NULL;
}
n *= 2;
}
meth_state *m_state = PyType_GetModuleState(defining_class);
m_state->counter += n;
Py_RETURN_NONE;
}
/*[clinic input]
_testmultiphase.StateAccessType.get_count
cls: defining_class
Return the value of the module-state counter.
[clinic start generated code]*/
static PyObject *
_testmultiphase_StateAccessType_get_count_impl(StateAccessTypeObject *self,
PyTypeObject *cls)
/*[clinic end generated code: output=64600f95b499a319 input=d5d181f12384849f]*/
{
meth_state *m_state = PyType_GetModuleState(cls);
return PyLong_FromLong(m_state->counter);
}
static PyMethodDef StateAccessType_methods[] = {
_TESTMULTIPHASE_STATEACCESSTYPE_GET_DEFINING_MODULE_METHODDEF
_TESTMULTIPHASE_STATEACCESSTYPE_GET_COUNT_METHODDEF
_TESTMULTIPHASE_STATEACCESSTYPE_INCREMENT_COUNT_CLINIC_METHODDEF
{
"increment_count_noclinic",
(PyCFunction)(void(*)(void))_StateAccessType_increment_count_noclinic,
METH_METHOD|METH_FASTCALL|METH_KEYWORDS,
_StateAccessType_decrement_count__doc__
},
{NULL, NULL} /* sentinel */
};
static PyType_Slot StateAccessType_Type_slots[] = {
{Py_tp_doc, "Type for testing per-module state access from methods."},
{Py_tp_methods, StateAccessType_methods},
{0, NULL}
};
static PyType_Spec StateAccessType_spec = {
"_testimportexec.StateAccessType",
sizeof(StateAccessTypeObject),
0,
Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_FINALIZE | Py_TPFLAGS_BASETYPE,
StateAccessType_Type_slots
};
/* Function of two integers returning integer */
PyDoc_STRVAR(testexport_foo_doc,
@ -193,30 +355,39 @@ static int execfunc(PyObject *m)
/* Add a custom type */
temp = PyType_FromSpec(&Example_Type_spec);
if (temp == NULL)
if (temp == NULL) {
goto fail;
if (PyModule_AddObject(m, "Example", temp) != 0)
}
if (PyModule_AddObject(m, "Example", temp) != 0) {
goto fail;
}
/* Add an exception type */
temp = PyErr_NewException("_testimportexec.error", NULL, NULL);
if (temp == NULL)
if (temp == NULL) {
goto fail;
if (PyModule_AddObject(m, "error", temp) != 0)
}
if (PyModule_AddObject(m, "error", temp) != 0) {
goto fail;
}
/* Add Str */
temp = PyType_FromSpec(&Str_Type_spec);
if (temp == NULL)
if (temp == NULL) {
goto fail;
if (PyModule_AddObject(m, "Str", temp) != 0)
}
if (PyModule_AddObject(m, "Str", temp) != 0) {
goto fail;
}
if (PyModule_AddIntConstant(m, "int_const", 1969) != 0)
if (PyModule_AddIntConstant(m, "int_const", 1969) != 0) {
goto fail;
}
if (PyModule_AddStringConstant(m, "str_const", "something different") != 0)
if (PyModule_AddStringConstant(m, "str_const", "something different") != 0) {
goto fail;
}
return 0;
fail:
@ -620,6 +791,54 @@ PyInit__testmultiphase_exec_unreported_exception(PyObject *spec)
return PyModuleDef_Init(&def_exec_unreported_exception);
}
static int
meth_state_access_exec(PyObject *m)
{
PyObject *temp;
meth_state *m_state;
m_state = PyModule_GetState(m);
if (m_state == NULL) {
return -1;
}
temp = PyType_FromModuleAndSpec(m, &StateAccessType_spec, NULL);
if (temp == NULL) {
return -1;
}
if (PyModule_AddObject(m, "StateAccessType", temp) != 0) {
return -1;
}
return 0;
}
static PyModuleDef_Slot meth_state_access_slots[] = {
{Py_mod_exec, meth_state_access_exec},
{0, NULL}
};
static PyModuleDef def_meth_state_access = {
PyModuleDef_HEAD_INIT, /* m_base */
"_testmultiphase_meth_state_access", /* m_name */
PyDoc_STR("Module testing access"
" to state from methods."),
sizeof(meth_state), /* m_size */
NULL, /* m_methods */
meth_state_access_slots, /* m_slots */
0, /* m_traverse */
0, /* m_clear */
0, /* m_free */
};
PyMODINIT_FUNC
PyInit__testmultiphase_meth_state_access(PyObject *spec)
{
return PyModuleDef_Init(&def_meth_state_access);
}
/*** Helper for imp test ***/
static PyModuleDef imp_dummy_def = TEST_MODULE_DEF("imp_dummy", main_slots, testexport_methods);

101
Modules/clinic/_testmultiphase.c.h generated Normal file
View file

@ -0,0 +1,101 @@
/*[clinic input]
preserve
[clinic start generated code]*/
PyDoc_STRVAR(_testmultiphase_StateAccessType_get_defining_module__doc__,
"get_defining_module($self, /)\n"
"--\n"
"\n"
"Return the module of the defining class.");
#define _TESTMULTIPHASE_STATEACCESSTYPE_GET_DEFINING_MODULE_METHODDEF \
{"get_defining_module", (PyCFunction)(void(*)(void))_testmultiphase_StateAccessType_get_defining_module, METH_METHOD|METH_FASTCALL|METH_KEYWORDS, _testmultiphase_StateAccessType_get_defining_module__doc__},
static PyObject *
_testmultiphase_StateAccessType_get_defining_module_impl(StateAccessTypeObject *self,
PyTypeObject *cls);
static PyObject *
_testmultiphase_StateAccessType_get_defining_module(StateAccessTypeObject *self, PyTypeObject *cls, PyObject *const *args, Py_ssize_t nargs, PyObject *kwnames)
{
PyObject *return_value = NULL;
static const char * const _keywords[] = { NULL};
static _PyArg_Parser _parser = {":get_defining_module", _keywords, 0};
if (!_PyArg_ParseStackAndKeywords(args, nargs, kwnames, &_parser
)) {
goto exit;
}
return_value = _testmultiphase_StateAccessType_get_defining_module_impl(self, cls);
exit:
return return_value;
}
PyDoc_STRVAR(_testmultiphase_StateAccessType_increment_count_clinic__doc__,
"increment_count_clinic($self, /, n=1, *, twice=False)\n"
"--\n"
"\n"
"Add \'n\' from the module-state counter.\n"
"\n"
"Pass \'twice\' to double that amount.\n"
"\n"
"This tests Argument Clinic support for defining_class.");
#define _TESTMULTIPHASE_STATEACCESSTYPE_INCREMENT_COUNT_CLINIC_METHODDEF \
{"increment_count_clinic", (PyCFunction)(void(*)(void))_testmultiphase_StateAccessType_increment_count_clinic, METH_METHOD|METH_FASTCALL|METH_KEYWORDS, _testmultiphase_StateAccessType_increment_count_clinic__doc__},
static PyObject *
_testmultiphase_StateAccessType_increment_count_clinic_impl(StateAccessTypeObject *self,
PyTypeObject *cls,
int n, int twice);
static PyObject *
_testmultiphase_StateAccessType_increment_count_clinic(StateAccessTypeObject *self, PyTypeObject *cls, PyObject *const *args, Py_ssize_t nargs, PyObject *kwnames)
{
PyObject *return_value = NULL;
static const char * const _keywords[] = {"n", "twice", NULL};
static _PyArg_Parser _parser = {"|i$p:increment_count_clinic", _keywords, 0};
int n = 1;
int twice = 0;
if (!_PyArg_ParseStackAndKeywords(args, nargs, kwnames, &_parser,
&n, &twice)) {
goto exit;
}
return_value = _testmultiphase_StateAccessType_increment_count_clinic_impl(self, cls, n, twice);
exit:
return return_value;
}
PyDoc_STRVAR(_testmultiphase_StateAccessType_get_count__doc__,
"get_count($self, /)\n"
"--\n"
"\n"
"Return the value of the module-state counter.");
#define _TESTMULTIPHASE_STATEACCESSTYPE_GET_COUNT_METHODDEF \
{"get_count", (PyCFunction)(void(*)(void))_testmultiphase_StateAccessType_get_count, METH_METHOD|METH_FASTCALL|METH_KEYWORDS, _testmultiphase_StateAccessType_get_count__doc__},
static PyObject *
_testmultiphase_StateAccessType_get_count_impl(StateAccessTypeObject *self,
PyTypeObject *cls);
static PyObject *
_testmultiphase_StateAccessType_get_count(StateAccessTypeObject *self, PyTypeObject *cls, PyObject *const *args, Py_ssize_t nargs, PyObject *kwnames)
{
PyObject *return_value = NULL;
static const char * const _keywords[] = { NULL};
static _PyArg_Parser _parser = {":get_count", _keywords, 0};
if (!_PyArg_ParseStackAndKeywords(args, nargs, kwnames, &_parser
)) {
goto exit;
}
return_value = _testmultiphase_StateAccessType_get_count_impl(self, cls);
exit:
return return_value;
}
/*[clinic end generated code: output=39eea487e94e7f5d input=a9049054013a1b77]*/

View file

@ -127,7 +127,11 @@ classmethod_get(PyMethodDescrObject *descr, PyObject *obj, PyObject *type)
((PyTypeObject *)type)->tp_name);
return NULL;
}
return PyCFunction_NewEx(descr->d_method, type, NULL);
PyTypeObject *cls = NULL;
if (descr->d_method->ml_flags & METH_METHOD) {
cls = descr->d_common.d_type;
}
return PyCMethod_New(descr->d_method, type, NULL, cls);
}
static PyObject *
@ -137,7 +141,19 @@ method_get(PyMethodDescrObject *descr, PyObject *obj, PyObject *type)
if (descr_check((PyDescrObject *)descr, obj, &res))
return res;
return PyCFunction_NewEx(descr->d_method, obj, NULL);
if (descr->d_method->ml_flags & METH_METHOD) {
if (PyType_Check(type)) {
return PyCMethod_New(descr->d_method, obj, NULL, descr->d_common.d_type);
} else {
PyErr_Format(PyExc_TypeError,
"descriptor '%V' needs a type, not '%s', as arg 2",
descr_name((PyDescrObject *)descr),
Py_TYPE(type)->tp_name);
return NULL;
}
} else {
return PyCFunction_NewEx(descr->d_method, obj, NULL);
}
}
static PyObject *
@ -335,6 +351,27 @@ method_vectorcall_VARARGS_KEYWORDS(
return result;
}
static PyObject *
method_vectorcall_FASTCALL_KEYWORDS_METHOD(
PyObject *func, PyObject *const *args, size_t nargsf, PyObject *kwnames)
{
PyThreadState *tstate = _PyThreadState_GET();
Py_ssize_t nargs = PyVectorcall_NARGS(nargsf);
if (method_check_args(func, args, nargs, NULL)) {
return NULL;
}
NULL;
PyCMethod meth = (PyCMethod) method_enter_call(tstate, func);
if (meth == NULL) {
return NULL;
}
PyObject *result = meth(args[0],
((PyMethodDescrObject *)func)->d_common.d_type,
args+1, nargs-1, kwnames);
Py_LeaveRecursiveCall();
return result;
}
static PyObject *
method_vectorcall_FASTCALL(
PyObject *func, PyObject *const *args, size_t nargsf, PyObject *kwnames)
@ -868,7 +905,8 @@ PyDescr_NewMethod(PyTypeObject *type, PyMethodDef *method)
{
/* Figure out correct vectorcall function to use */
vectorcallfunc vectorcall;
switch (method->ml_flags & (METH_VARARGS | METH_FASTCALL | METH_NOARGS | METH_O | METH_KEYWORDS))
switch (method->ml_flags & (METH_VARARGS | METH_FASTCALL | METH_NOARGS |
METH_O | METH_KEYWORDS | METH_METHOD))
{
case METH_VARARGS:
vectorcall = method_vectorcall_VARARGS;
@ -888,6 +926,9 @@ PyDescr_NewMethod(PyTypeObject *type, PyMethodDef *method)
case METH_O:
vectorcall = method_vectorcall_O;
break;
case METH_METHOD | METH_FASTCALL | METH_KEYWORDS:
vectorcall = method_vectorcall_FASTCALL_KEYWORDS_METHOD;
break;
default:
PyErr_Format(PyExc_SystemError,
"%s() method: bad call flags", method->ml_name);

View file

@ -10,12 +10,16 @@
/* undefine macro trampoline to PyCFunction_NewEx */
#undef PyCFunction_New
/* undefine macro trampoline to PyCMethod_New */
#undef PyCFunction_NewEx
/* Forward declarations */
static PyObject * cfunction_vectorcall_FASTCALL(
PyObject *func, PyObject *const *args, size_t nargsf, PyObject *kwnames);
static PyObject * cfunction_vectorcall_FASTCALL_KEYWORDS(
PyObject *func, PyObject *const *args, size_t nargsf, PyObject *kwnames);
static PyObject * cfunction_vectorcall_FASTCALL_KEYWORDS_METHOD(
PyObject *func, PyObject *const *args, size_t nargsf, PyObject *kwnames);
static PyObject * cfunction_vectorcall_NOARGS(
PyObject *func, PyObject *const *args, size_t nargsf, PyObject *kwnames);
static PyObject * cfunction_vectorcall_O(
@ -32,10 +36,17 @@ PyCFunction_New(PyMethodDef *ml, PyObject *self)
PyObject *
PyCFunction_NewEx(PyMethodDef *ml, PyObject *self, PyObject *module)
{
return PyCMethod_New(ml, self, module, NULL);
}
PyObject *
PyCMethod_New(PyMethodDef *ml, PyObject *self, PyObject *module, PyTypeObject *cls)
{
/* Figure out correct vectorcall function to use */
vectorcallfunc vectorcall;
switch (ml->ml_flags & (METH_VARARGS | METH_FASTCALL | METH_NOARGS | METH_O | METH_KEYWORDS))
switch (ml->ml_flags & (METH_VARARGS | METH_FASTCALL | METH_NOARGS |
METH_O | METH_KEYWORDS | METH_METHOD))
{
case METH_VARARGS:
case METH_VARARGS | METH_KEYWORDS:
@ -55,17 +66,44 @@ PyCFunction_NewEx(PyMethodDef *ml, PyObject *self, PyObject *module)
case METH_O:
vectorcall = cfunction_vectorcall_O;
break;
case METH_METHOD | METH_FASTCALL | METH_KEYWORDS:
vectorcall = cfunction_vectorcall_FASTCALL_KEYWORDS_METHOD;
break;
default:
PyErr_Format(PyExc_SystemError,
"%s() method: bad call flags", ml->ml_name);
return NULL;
}
PyCFunctionObject *op =
PyObject_GC_New(PyCFunctionObject, &PyCFunction_Type);
if (op == NULL) {
return NULL;
PyCFunctionObject *op = NULL;
if (ml->ml_flags & METH_METHOD) {
if (!cls) {
PyErr_SetString(PyExc_SystemError,
"attempting to create PyCMethod with a METH_METHOD "
"flag but no class");
return NULL;
}
PyCMethodObject *om = PyObject_GC_New(PyCMethodObject, &PyCMethod_Type);
if (om == NULL) {
return NULL;
}
Py_INCREF(cls);
om->mm_class = cls;
op = (PyCFunctionObject *)om;
} else {
if (cls) {
PyErr_SetString(PyExc_SystemError,
"attempting to create PyCFunction with class "
"but no METH_METHOD flag");
return NULL;
}
op = PyObject_GC_New(PyCFunctionObject, &PyCFunction_Type);
if (op == NULL) {
return NULL;
}
}
op->m_weakreflist = NULL;
op->m_ml = ml;
Py_XINCREF(self);
@ -107,6 +145,16 @@ PyCFunction_GetFlags(PyObject *op)
return PyCFunction_GET_FLAGS(op);
}
PyTypeObject *
PyCMethod_GetClass(PyObject *op)
{
if (!PyCFunction_Check(op)) {
PyErr_BadInternalCall();
return NULL;
}
return PyCFunction_GET_CLASS(op);
}
/* Methods (the standard built-in methods, that is) */
static void
@ -118,6 +166,7 @@ meth_dealloc(PyCFunctionObject *m)
}
Py_XDECREF(m->m_self);
Py_XDECREF(m->m_module);
Py_XDECREF(PyCFunction_GET_CLASS(m));
PyObject_GC_Del(m);
}
@ -196,6 +245,7 @@ meth_traverse(PyCFunctionObject *m, visitproc visit, void *arg)
{
Py_VISIT(m->m_self);
Py_VISIT(m->m_module);
Py_VISIT(PyCFunction_GET_CLASS(m));
return 0;
}
@ -314,6 +364,13 @@ PyTypeObject PyCFunction_Type = {
0, /* tp_dict */
};
PyTypeObject PyCMethod_Type = {
PyVarObject_HEAD_INIT(&PyType_Type, 0)
.tp_name = "builtin_method",
.tp_basicsize = sizeof(PyCMethodObject),
.tp_base = &PyCFunction_Type,
};
/* Vectorcall functions for each of the PyCFunction calling conventions,
* except for METH_VARARGS (possibly combined with METH_KEYWORDS) which
* doesn't use vectorcall.
@ -385,6 +442,22 @@ cfunction_vectorcall_FASTCALL_KEYWORDS(
return result;
}
static PyObject *
cfunction_vectorcall_FASTCALL_KEYWORDS_METHOD(
PyObject *func, PyObject *const *args, size_t nargsf, PyObject *kwnames)
{
PyThreadState *tstate = _PyThreadState_GET();
PyTypeObject *cls = PyCFunction_GET_CLASS(func);
Py_ssize_t nargs = PyVectorcall_NARGS(nargsf);
PyCMethod meth = (PyCMethod)cfunction_enter_call(tstate, func);
if (meth == NULL) {
return NULL;
}
PyObject *result = meth(PyCFunction_GET_SELF(func), cls, args, nargs, kwnames);
_Py_LeaveRecursiveCall(tstate);
return result;
}
static PyObject *
cfunction_vectorcall_NOARGS(
PyObject *func, PyObject *const *args, size_t nargsf, PyObject *kwnames)

View file

@ -1789,6 +1789,7 @@ _PyTypes_Init(void)
INIT_TYPE(&PyCode_Type, "code");
INIT_TYPE(&PyFrame_Type, "frame");
INIT_TYPE(&PyCFunction_Type, "builtin function");
INIT_TYPE(&PyCMethod_Type, "builtin method");
INIT_TYPE(&PyMethod_Type, "method");
INIT_TYPE(&PyFunction_Type, "function");
INIT_TYPE(&PyDictProxy_Type, "dict proxy");

View file

@ -2708,6 +2708,9 @@ type_new(PyTypeObject *metatype, PyObject *args, PyObject *kwds)
if (qualname != NULL && _PyDict_DelItemId(dict, &PyId___qualname__) < 0)
goto error;
/* Set ht_module */
et->ht_module = NULL;
/* Set tp_doc to a copy of dict['__doc__'], if the latter is there
and is a string. The __doc__ accessor will first look for tp_doc;
if that fails, it will still look into __dict__.
@ -2939,6 +2942,12 @@ PyType_FromSpec_tp_traverse(PyObject *self, visitproc visit, void *arg)
PyObject *
PyType_FromSpecWithBases(PyType_Spec *spec, PyObject *bases)
{
return PyType_FromModuleAndSpec(NULL, spec, bases);
}
PyObject *
PyType_FromModuleAndSpec(PyObject *module, PyType_Spec *spec, PyObject *bases)
{
PyHeapTypeObject *res;
PyObject *modname;
@ -2998,6 +3007,9 @@ PyType_FromSpecWithBases(PyType_Spec *spec, PyObject *bases)
Py_INCREF(res->ht_qualname);
type->tp_name = spec->name;
Py_XINCREF(module);
res->ht_module = module;
/* Adjust for empty tuple bases */
if (!bases) {
base = &PyBaseObject_Type;
@ -3176,6 +3188,40 @@ PyType_GetSlot(PyTypeObject *type, int slot)
return *(void**)(((char*)type) + slotoffsets[slot]);
}
PyObject *
PyType_GetModule(PyTypeObject *type)
{
assert(PyType_Check(type));
if (!_PyType_HasFeature(type, Py_TPFLAGS_HEAPTYPE)) {
PyErr_Format(
PyExc_TypeError,
"PyType_GetModule: Type '%s' is not a heap type",
type->tp_name);
return NULL;
}
PyHeapTypeObject* et = (PyHeapTypeObject*)type;
if (!et->ht_module) {
PyErr_Format(
PyExc_TypeError,
"PyType_GetModule: Type '%s' has no associated module",
type->tp_name);
return NULL;
}
return et->ht_module;
}
void *
PyType_GetModuleState(PyTypeObject *type)
{
PyObject *m = PyType_GetModule(type);
if (m == NULL) {
return NULL;
}
return PyModule_GetState(m);
}
/* Internal API to look for a name through the MRO, bypassing the method cache.
This returns a borrowed reference, and might set an exception.
'error' is set to: -1: error with exception; 1: error without exception; 0: ok */
@ -3503,8 +3549,10 @@ type_dealloc(PyTypeObject *type)
Py_XDECREF(et->ht_name);
Py_XDECREF(et->ht_qualname);
Py_XDECREF(et->ht_slots);
if (et->ht_cached_keys)
if (et->ht_cached_keys) {
_PyDictKeys_DecRef(et->ht_cached_keys);
}
Py_XDECREF(et->ht_module);
Py_TYPE(type)->tp_free((PyObject *)type);
}
@ -3694,6 +3742,7 @@ type_traverse(PyTypeObject *type, visitproc visit, void *arg)
Py_VISIT(type->tp_mro);
Py_VISIT(type->tp_bases);
Py_VISIT(type->tp_base);
Py_VISIT(((PyHeapTypeObject *)type)->ht_module);
/* There's no need to visit type->tp_subclasses or
((PyHeapTypeObject *)type)->ht_slots, because they can't be involved
@ -3715,10 +3764,13 @@ type_clear(PyTypeObject *type)
the dict, so that other objects caught in a reference cycle
don't start calling destroyed methods.
Otherwise, the only field we need to clear is tp_mro, which is
Otherwise, the we need to clear tp_mro, which is
part of a hard cycle (its first element is the class itself) that
won't be broken otherwise (it's a tuple and tuples don't have a
tp_clear handler). None of the other fields need to be
tp_clear handler).
We also need to clear ht_module, if present: the module usually holds a
reference to its class. None of the other fields need to be
cleared, and here's why:
tp_cache:
@ -3743,8 +3795,11 @@ type_clear(PyTypeObject *type)
((PyHeapTypeObject *)type)->ht_cached_keys = NULL;
_PyDictKeys_DecRef(cached_keys);
}
if (type->tp_dict)
if (type->tp_dict) {
PyDict_Clear(type->tp_dict);
}
Py_CLEAR(((PyHeapTypeObject *)type)->ht_module);
Py_CLEAR(type->tp_mro);
return 0;

View file

@ -138,6 +138,7 @@
<ClInclude Include="..\Include\cpython\import.h" />
<ClInclude Include="..\Include\cpython\initconfig.h" />
<ClInclude Include="..\Include\cpython\listobject.h" />
<ClInclude Include="..\Include\cpython\methodobject.h" />
<ClInclude Include="..\Include\cpython\object.h" />
<ClInclude Include="..\Include\cpython\objimpl.h" />
<ClInclude Include="..\Include\cpython\pyerrors.h" />

View file

@ -111,6 +111,9 @@
<ClInclude Include="..\Include\cpython\listobject.h">
<Filter>Include</Filter>
</ClInclude>
<ClInclude Include="..\Include\cpython\methodobject.h">
<Filter>Include</Filter>
</ClInclude>
<ClInclude Include="..\Include\cpython\object.h">
<Filter>Include</Filter>
</ClInclude>

View file

@ -657,9 +657,14 @@ def output_templates(self, f):
if not p.is_optional():
min_pos = i
requires_defining_class = any(
isinstance(p.converter, defining_class_converter)
for p in parameters)
meth_o = (len(parameters) == 1 and
parameters[0].is_positional_only() and
not converters[0].is_optional() and
not requires_defining_class and
not new_or_init)
# we have to set these things before we're done:
@ -717,6 +722,11 @@ def output_templates(self, f):
{c_basename}({self_type}{self_name}, PyObject *const *args, Py_ssize_t nargs, PyObject *kwnames)
""")
parser_prototype_def_class = normalize_snippet("""
static PyObject *
{c_basename}({self_type}{self_name}, PyTypeObject *cls, PyObject *const *args, Py_ssize_t nargs, PyObject *kwnames)
""")
# parser_body_fields remembers the fields passed in to the
# previous call to parser_body. this is used for an awful hack.
parser_body_fields = ()
@ -824,7 +834,7 @@ def parser_body(prototype, *fields, declarations=''):
parser_definition = parser_body(parser_prototype, ' {option_group_parsing}')
elif pos_only == len(parameters):
elif not requires_defining_class and pos_only == len(parameters):
if not new_or_init:
# positional-only, but no option groups
# we only need one call to _PyArg_ParseStack
@ -891,7 +901,7 @@ def parser_body(prototype, *fields, declarations=''):
parser_prototype = parser_prototype_fastcall_keywords
argname_fmt = 'args[%d]'
declarations = normalize_snippet("""
static const char * const _keywords[] = {{{keywords}, NULL}};
static const char * const _keywords[] = {{{keywords} NULL}};
static _PyArg_Parser _parser = {{NULL, _keywords, "{name}", 0}};
PyObject *argsbuf[%s];
""" % len(converters))
@ -909,7 +919,7 @@ def parser_body(prototype, *fields, declarations=''):
parser_prototype = parser_prototype_keyword
argname_fmt = 'fastargs[%d]'
declarations = normalize_snippet("""
static const char * const _keywords[] = {{{keywords}, NULL}};
static const char * const _keywords[] = {{{keywords} NULL}};
static _PyArg_Parser _parser = {{NULL, _keywords, "{name}", 0}};
PyObject *argsbuf[%s];
PyObject * const *fastargs;
@ -923,6 +933,9 @@ def parser_body(prototype, *fields, declarations=''):
goto exit;
}}
""" % (min_pos, max_pos, min_kw_only), indent=4)]
if requires_defining_class:
flags = 'METH_METHOD|' + flags
parser_prototype = parser_prototype_def_class
add_label = None
for i, p in enumerate(parameters):
@ -983,11 +996,11 @@ def parser_body(prototype, *fields, declarations=''):
parser_code.append("%s:" % add_label)
else:
declarations = (
'static const char * const _keywords[] = {{{keywords}, NULL}};\n'
'static const char * const _keywords[] = {{{keywords} NULL}};\n'
'static _PyArg_Parser _parser = {{"{format_units}:{name}", _keywords, 0}};')
if not new_or_init:
parser_code = [normalize_snippet("""
if (!_PyArg_ParseStackAndKeywords(args, nargs, kwnames, &_parser,
if (!_PyArg_ParseStackAndKeywords(args, nargs, kwnames, &_parser{parse_arguments_comma}
{parse_arguments})) {{
goto exit;
}}
@ -1021,6 +1034,9 @@ def parser_body(prototype, *fields, declarations=''):
if parses_keywords:
assert parses_positional
if requires_defining_class:
raise ValueError("Slot methods cannot access their defining class.")
if not parses_keywords:
fields.insert(0, normalize_snippet("""
if ({self_type_check}!_PyArg_NoKeywords("{name}", kwargs)) {{
@ -1297,9 +1313,13 @@ def render_function(self, clinic, f):
template_dict['declarations'] = format_escape("\n".join(data.declarations))
template_dict['initializers'] = "\n\n".join(data.initializers)
template_dict['modifications'] = '\n\n'.join(data.modifications)
template_dict['keywords'] = '"' + '", "'.join(data.keywords) + '"'
template_dict['keywords'] = ' '.join('"' + k + '",' for k in data.keywords)
template_dict['format_units'] = ''.join(data.format_units)
template_dict['parse_arguments'] = ', '.join(data.parse_arguments)
if data.parse_arguments:
template_dict['parse_arguments_comma'] = ',';
else:
template_dict['parse_arguments_comma'] = '';
template_dict['impl_parameters'] = ", ".join(data.impl_parameters)
template_dict['impl_arguments'] = ", ".join(data.impl_arguments)
template_dict['return_conversion'] = format_escape("".join(data.return_conversion).rstrip())
@ -2730,6 +2750,25 @@ def parse_arg(self, argname, displayname):
""".format(argname=argname, paramname=self.name)
return super().parse_arg(argname, displayname)
class defining_class_converter(CConverter):
"""
A special-case converter:
this is the default converter used for the defining class.
"""
type = 'PyTypeObject *'
format_unit = ''
show_in_signature = False
def converter_init(self, *, type=None):
self.specified_type = type
def render(self, parameter, data):
self._render_self(parameter, data)
def set_template_dict(self, template_dict):
template_dict['defining_class_name'] = self.name
class char_converter(CConverter):
type = 'char'
default_type = (bytes, bytearray)
@ -4508,6 +4547,19 @@ def bad_node(self, node):
else:
fail("A 'self' parameter, if specified, must be the very first thing in the parameter block.")
if isinstance(converter, defining_class_converter):
_lp = len(self.function.parameters)
if _lp == 1:
if (self.parameter_state != self.ps_required):
fail("A 'defining_class' parameter cannot be marked optional.")
if value is not unspecified:
fail("A 'defining_class' parameter cannot have a default value.")
if self.group:
fail("A 'defining_class' parameter cannot be in an optional group.")
else:
fail("A 'defining_class' parameter, if specified, must either be the first thing in the parameter block, or come just after 'self'.")
p = Parameter(parameter_name, kind, function=self.function, converter=converter, default=value, group=self.group)
if parameter_name in self.function.parameters: