languages/cpython
languages/cpython
1
0
Fork 0
mirror of https://github.com/python/cpython synced 2024-10-14 08:51:20 +00:00
Code Issues Packages Projects Releases Wiki Activity

bpo-43014: Improve performance of tokenize.tokenize by 20-30%

Browse source
This commit is contained in:
Anthony Sottile 2021-01-24 01:23:17 -08:00 committed by GitHub
parent bf9239bb61
commit 15bd9efd01
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
2 changed files with 3 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
Lib/tokenize.py
View file

@ -27,6 +27,7 @@
from builtins import open as _builtin_open
from codecs import lookup, BOM_UTF8
import collections
import functools
from io import TextIOWrapper
import itertools as _itertools
import re
@ -95,6 +96,7 @@ def _all_string_prefixes():
result.add(''.join(u))
return result
@functools.lru_cache
def _compile(expr):
return re.compile(expr, re.UNICODE)

1
Misc/NEWS.d/next/Library/2021-01-24-00-37-40.bpo-43014.BVPhEr.rst Normal file
View file

@ -0,0 +1 @@
Improve performance of :mod:`tokenize` by 20-30%. Patch by Anthony Sottile.