![Mailman 3 [Python-checkins] gh-104584: Baby steps towards generating and executing traces (#105924) - Python-checkins (1)](https://i0.wp.com/secure.gravatar.com/avatar/cc7737cd64a84f1b5c61a160798e97ee.jpg?s=120&d=mm&r=g "Mailman 3 [Python-checkins] gh-104584: Baby steps towards generating and executing traces (#105924) - Python-checkins (1)")
27 Jun 2023 27 Jun '23
4:03 a.m.
https://github.com/python/cpython/commit/51fc72511733353de15bc633a3d7b6da366...commit: 51fc72511733353de15bc633a3d7b6da366842e4branch: mainauthor: Guido van Rossum <guido@python.org>committer: gvanrossum <gvanrossum@gmail.com>date: 2023-06-26T19:02:57-07:00summary:gh-104584: Baby steps towards generating and executing traces (#105924)Added a new, experimental, tracing optimizer and interpreter (a.k.a. "tier 2"). This currently pessimizes, so don't use yet -- this is infrastructure so we can experiment with optimizing passes. To enable it, pass ``-Xuops`` or set ``PYTHONUOPS=1``. To get debug output, set ``PYTHONUOPSDEBUG=N`` where ``N`` is a debug level (0-4, where 0 is no debug output and 4 is excessively verbose).All of this code is likely to change dramatically before the 3.13 feature freeze. But this is a first step.files:A Include/internal/pycore_uops.hA Misc/NEWS.d/next/Core and Builtins/2023-06-27-00-58-26.gh-issue-104584.Wu-uXy.rstA Python/executor_cases.c.hM .gitattributesM Include/cpython/optimizer.hM Include/pystats.hM Makefile.pre.inM Modules/_testinternalcapi.cM Python/bytecodes.cM Python/ceval.cM Python/ceval_macros.hM Python/generated_cases.c.hM Python/opcode_metadata.hM Python/optimizer.cM Python/pylifecycle.cM Python/specialize.cM Tools/c-analyzer/cpython/_parser.pyM Tools/c-analyzer/cpython/ignored.tsvM Tools/cases_generator/README.mdM Tools/cases_generator/generate_cases.pyM Tools/cases_generator/test_generator.pydiff --git a/.gitattributes b/.gitattributesindex bab1ef0d01046..a2ff14c66323e 100644--- a/.gitattributes+++ b/.gitattributes@@ -86,6 +86,7 @@ Parser/token.c generated Programs/test_frozenmain.h generated Python/Python-ast.c generated Python/generated_cases.c.h generated+Python/executor_cases.c.h generated Python/opcode_targets.h generated Python/stdlib_module_names.h generated Tools/peg_generator/pegen/grammar_parser.py generateddiff --git a/Include/cpython/optimizer.h b/Include/cpython/optimizer.hindex b2d173fb913ee..2664f5bc4b174 100644--- a/Include/cpython/optimizer.h+++ b/Include/cpython/optimizer.h@@ -45,6 +45,7 @@ extern _PyOptimizerObject _PyOptimizer_Default; /* For testing */ PyAPI_FUNC(PyObject *)PyUnstable_Optimizer_NewCounter(void);+PyAPI_FUNC(PyObject *)PyUnstable_Optimizer_NewUOpOptimizer(void); #define OPTIMIZER_BITS_IN_COUNTER 4 diff --git a/Include/internal/pycore_uops.h b/Include/internal/pycore_uops.hnew file mode 100644index 0000000000000..0e88d7e7f4a3b--- /dev/null+++ b/Include/internal/pycore_uops.h@@ -0,0 +1,31 @@+#ifndef Py_INTERNAL_UOPS_H+#define Py_INTERNAL_UOPS_H+#ifdef __cplusplus+extern "C" {+#endif++#ifndef Py_BUILD_CORE+# error "this header requires Py_BUILD_CORE define"+#endif++#define _Py_UOP_MAX_TRACE_LENGTH 16++typedef struct {+ int opcode;+ uint64_t operand; // Sometimes oparg, sometimes a cache entry+} _PyUOpInstruction;++typedef struct {+ _PyExecutorObject base;+ _PyUOpInstruction trace[_Py_UOP_MAX_TRACE_LENGTH]; // TODO: variable length+} _PyUOpExecutorObject;++_PyInterpreterFrame *_PyUopExecute(+ _PyExecutorObject *executor,+ _PyInterpreterFrame *frame,+ PyObject **stack_pointer);++#ifdef __cplusplus+}+#endif+#endif /* !Py_INTERNAL_UOPS_H */diff --git a/Include/pystats.h b/Include/pystats.hindex 034bf05bfe290..54c9b8d8b3538 100644--- a/Include/pystats.h+++ b/Include/pystats.h@@ -71,6 +71,9 @@ typedef struct _object_stats { uint64_t type_cache_dunder_misses; uint64_t type_cache_collisions; uint64_t optimization_attempts;+ uint64_t optimization_traces_created;+ uint64_t optimization_traces_executed;+ uint64_t optimization_uops_executed; } ObjectStats; typedef struct _stats {diff --git a/Makefile.pre.in b/Makefile.pre.inindex e9a8d8ffb71fd..98b58cf1299f1 100644--- a/Makefile.pre.in+++ b/Makefile.pre.in@@ -1542,19 +1542,9 @@ regen-opcode-targets: .PHONY: regen-cases regen-cases:-# Regenerate Python/generated_cases.c.h-# and Python/opcode_metadata.h-# from Python/bytecodes.c-# using Tools/cases_generator/generate_cases.py+# Regenerate various files from Python/bytecodes.c PYTHONPATH=$(srcdir)/Tools/cases_generator \-$(PYTHON_FOR_REGEN) \- $(srcdir)/Tools/cases_generator/generate_cases.py \---emit-line-directives \--o $(srcdir)/Python/generated_cases.c.h.new \--m $(srcdir)/Python/opcode_metadata.h.new \-$(srcdir)/Python/bytecodes.c-$(UPDATE_FILE) $(srcdir)/Python/generated_cases.c.h $(srcdir)/Python/generated_cases.c.h.new-$(UPDATE_FILE) $(srcdir)/Python/opcode_metadata.h $(srcdir)/Python/opcode_metadata.h.new+$(PYTHON_FOR_REGEN) $(srcdir)/Tools/cases_generator/generate_cases.py -l Python/compile.o: $(srcdir)/Python/opcode_metadata.h @@ -1565,6 +1555,13 @@ Python/ceval.o: \ $(srcdir)/Python/opcode_metadata.h \ $(srcdir)/Python/opcode_targets.h +Python/flowgraph.o: \+$(srcdir)/Python/opcode_metadata.h++Python/optimizer.o: \+$(srcdir)/Python/executor_cases.c.h \+$(srcdir)/Python/opcode_metadata.h+ Python/frozen.o: $(FROZEN_FILES_OUT) # Generate DTrace probe macros, then rename them (PYTHON_ -> PyDTrace_) todiff --git a/Misc/NEWS.d/next/Core and Builtins/2023-06-27-00-58-26.gh-issue-104584.Wu-uXy.rst b/Misc/NEWS.d/next/Core and Builtins/2023-06-27-00-58-26.gh-issue-104584.Wu-uXy.rstnew file mode 100644index 0000000000000..a36490104ba3a--- /dev/null+++ b/Misc/NEWS.d/next/Core and Builtins/2023-06-27-00-58-26.gh-issue-104584.Wu-uXy.rst@@ -0,0 +1 @@+Added a new, experimental, tracing optimizer and interpreter (a.k.a. "tier 2"). This currently pessimizes, so don't use yet -- this is infrastructure so we can experiment with optimizing passes. To enable it, pass ``-Xuops`` or set ``PYTHONUOPS=1``. To get debug output, set ``PYTHONUOPSDEBUG=N`` where ``N`` is a debug level (0-4, where 0 is no debug output and 4 is excessively verbose).diff --git a/Modules/_testinternalcapi.c b/Modules/_testinternalcapi.cindex 0a3b0dd539e86..3c0c2adaf6f84 100644--- a/Modules/_testinternalcapi.c+++ b/Modules/_testinternalcapi.c@@ -830,6 +830,12 @@ get_counter_optimizer(PyObject *self, PyObject *arg) return PyUnstable_Optimizer_NewCounter(); } +static PyObject *+get_uop_optimizer(PyObject *self, PyObject *arg)+{+ return PyUnstable_Optimizer_NewUOpOptimizer();+}+ static PyObject * set_optimizer(PyObject *self, PyObject *opt) {@@ -994,6 +1000,7 @@ static PyMethodDef module_functions[] = { {"get_optimizer", get_optimizer, METH_NOARGS, NULL}, {"set_optimizer", set_optimizer, METH_O, NULL}, {"get_counter_optimizer", get_counter_optimizer, METH_NOARGS, NULL},+ {"get_uop_optimizer", get_uop_optimizer, METH_NOARGS, NULL}, {"pending_threadfunc", _PyCFunction_CAST(pending_threadfunc), METH_VARARGS | METH_KEYWORDS}, // {"pending_fd_identify", pending_fd_identify, METH_VARARGS, NULL},diff --git a/Python/bytecodes.c b/Python/bytecodes.cindex 3ef8ed0ccaafd..02857104f5a4d 100644--- a/Python/bytecodes.c+++ b/Python/bytecodes.c@@ -52,8 +52,6 @@ #define family(name, ...) static int family_##name #define pseudo(name) static int pseudo_##name -typedef PyObject *(*convertion_func_ptr)(PyObject *);- // Dummy variables for stack effects. static PyObject *value, *value1, *value2, *left, *right, *res, *sum, *prod, *sub; static PyObject *container, *start, *stop, *v, *lhs, *rhs, *res2;@@ -2182,7 +2180,7 @@ dummy_func( frame = executor->execute(executor, frame, stack_pointer); if (frame == NULL) { frame = cframe.current_frame;- goto error;+ goto resume_with_error; } goto resume_frame; }diff --git a/Python/ceval.c b/Python/ceval.cindex 53107018978c0..3e0dcf8c8629b 100644--- a/Python/ceval.c+++ b/Python/ceval.c@@ -22,6 +22,7 @@ #include "pycore_sysmodule.h" // _PySys_Audit() #include "pycore_tuple.h" // _PyTuple_ITEMS() #include "pycore_typeobject.h" // _PySuper_Lookup()+#include "pycore_uops.h" // _PyUOpExecutorObject #include "pycore_emscripten_signal.h" // _Py_CHECK_EMSCRIPTEN_SIGNALS #include "pycore_dict.h"@@ -223,14 +224,6 @@ _PyEvalFramePushAndInit_Ex(PyThreadState *tstate, PyFunctionObject *func, static void _PyEvalFrameClearAndPop(PyThreadState *tstate, _PyInterpreterFrame *frame); -typedef PyObject *(*convertion_func_ptr)(PyObject *);--static const convertion_func_ptr CONVERSION_FUNCTIONS[4] = {- [FVC_STR] = PyObject_Str,- [FVC_REPR] = PyObject_Repr,- [FVC_ASCII] = PyObject_ASCII-};- #define UNBOUNDLOCAL_ERROR_MSG \ "cannot access local variable '%s' where it is not associated with a value" #define UNBOUNDFREE_ERROR_MSG \@@ -2771,3 +2764,131 @@ void Py_LeaveRecursiveCall(void) { _Py_LeaveRecursiveCall(); }++///////////////////// Experimental UOp Interpreter /////////////////////++// UPDATE_MISS_STATS (called by DEOPT_IF) uses next_instr+// TODO: Make it do something useful+#undef UPDATE_MISS_STATS+#define UPDATE_MISS_STATS(INSTNAME) ((void)0)++_PyInterpreterFrame *+_PyUopExecute(_PyExecutorObject *executor, _PyInterpreterFrame *frame, PyObject **stack_pointer)+{+#ifdef LLTRACE+ char *uop_debug = Py_GETENV("PYTHONUOPSDEBUG");+ int lltrace = 0;+ if (uop_debug != NULL && *uop_debug >= '0') {+ lltrace = *uop_debug - '0'; // TODO: Parse an int and all that+ }+ if (lltrace >= 2) {+ PyCodeObject *code = _PyFrame_GetCode(frame);+ _Py_CODEUNIT *instr = frame->prev_instr + 1;+ fprintf(stderr,+ "Entering _PyUopExecute for %s (%s:%d) at offset %ld\n",+ PyUnicode_AsUTF8(code->co_qualname),+ PyUnicode_AsUTF8(code->co_filename),+ code->co_firstlineno,+ (long)(instr - (_Py_CODEUNIT *)code->co_code_adaptive));+ }+#endif++ PyThreadState *tstate = _PyThreadState_GET();+ _PyUOpExecutorObject *self = (_PyUOpExecutorObject *)executor;++ // Equivalent to CHECK_EVAL_BREAKER()+ _Py_CHECK_EMSCRIPTEN_SIGNALS_PERIODICALLY();+ if (_Py_atomic_load_relaxed_int32(&tstate->interp->ceval.eval_breaker)) {+ if (_Py_HandlePending(tstate) != 0) {+ goto error;+ }+ }++ OBJECT_STAT_INC(optimization_traces_executed);+ _Py_CODEUNIT *ip_offset = (_Py_CODEUNIT *)_PyFrame_GetCode(frame)->co_code_adaptive - 1;+ int pc = 0;+ int opcode;+ uint64_t operand;+ int oparg;+ for (;;) {+ opcode = self->trace[pc].opcode;+ operand = self->trace[pc].operand;+ oparg = (int)operand;+#ifdef LLTRACE+ if (lltrace >= 3) {+ const char *opname = opcode < 256 ? _PyOpcode_OpName[opcode] : "";+ int stack_level = (int)(stack_pointer - _PyFrame_Stackbase(frame));+ fprintf(stderr, " uop %s %d, operand %" PRIu64 ", stack_level %d\n",+ opname, opcode, operand, stack_level);+ }+#endif+ pc++;+ OBJECT_STAT_INC(optimization_uops_executed);+ switch (opcode) {++#undef ENABLE_SPECIALIZATION+#define ENABLE_SPECIALIZATION 0+#include "executor_cases.c.h"++ case SET_IP:+ {+ frame->prev_instr = ip_offset + oparg;+ break;+ }++ case EXIT_TRACE:+ {+ _PyFrame_SetStackPointer(frame, stack_pointer);+ Py_DECREF(self);+ return frame;+ }++ default:+ {+ fprintf(stderr, "Unknown uop %d, operand %" PRIu64 "\n", opcode, operand);+ Py_FatalError("Unknown uop");+ abort(); // Unreachable+ for (;;) {}+ // Really unreachable+ }++ }+ }++pop_4_error:+ STACK_SHRINK(1);+pop_3_error:+ STACK_SHRINK(1);+pop_2_error:+ STACK_SHRINK(1);+pop_1_error:+ STACK_SHRINK(1);+error:+ // On ERROR_IF we return NULL as the frame.+ // The caller recovers the frame from cframe.current_frame.+#ifdef LLTRACE+ if (lltrace >= 2) {+ fprintf(stderr, "Error: [Opcode %d, operand %" PRIu64 "]\n", opcode, operand);+ }+#endif+ _PyFrame_SetStackPointer(frame, stack_pointer);+ Py_DECREF(self);+ return NULL;++PREDICTED(UNPACK_SEQUENCE)+PREDICTED(COMPARE_OP)+PREDICTED(LOAD_SUPER_ATTR)+PREDICTED(STORE_SUBSCR)+PREDICTED(BINARY_SUBSCR)+PREDICTED(BINARY_OP)+ // On DEOPT_IF we just repeat the last instruction.+ // This presumes nothing was popped from the stack (nor pushed).+#ifdef LLTRACE+ if (lltrace >= 2) {+ fprintf(stderr, "DEOPT: [Opcode %d, operand %" PRIu64 "]\n", opcode, operand);+ }+#endif+ _PyFrame_SetStackPointer(frame, stack_pointer);+ Py_DECREF(self);+ return frame;+}diff --git a/Python/ceval_macros.h b/Python/ceval_macros.hindex 706a9a24b26b0..0d41ef5a14cef 100644--- a/Python/ceval_macros.h+++ b/Python/ceval_macros.h@@ -1,4 +1,4 @@-// Macros needed by ceval.c and bytecodes.c+// Macros and other things needed by ceval.c and bytecodes.c /* Computed GOTOs, or the-optimization-commonly-but-improperly-known-as-"threaded code"@@ -339,3 +339,11 @@ do { \ goto error; \ } \ } while (0);++typedef PyObject *(*convertion_func_ptr)(PyObject *);++static const convertion_func_ptr CONVERSION_FUNCTIONS[4] = {+ [FVC_STR] = PyObject_Str,+ [FVC_REPR] = PyObject_Repr,+ [FVC_ASCII] = PyObject_ASCII+};diff --git a/Python/executor_cases.c.h b/Python/executor_cases.c.hnew file mode 100644index 0000000000000..1bdee40d4f8ee--- /dev/null+++ b/Python/executor_cases.c.h@@ -0,0 +1,1606 @@+// This file is generated by Tools/cases_generator/generate_cases.py+// from:+// Python/bytecodes.c+// Do not edit!++ case NOP: {+ break;+ }++ case LOAD_FAST: {+ PyObject *value;+ value = GETLOCAL(oparg);+ assert(value != NULL);+ Py_INCREF(value);+ STACK_GROW(1);+ stack_pointer[-1] = value;+ break;+ }++ case LOAD_FAST_AND_CLEAR: {+ PyObject *value;+ value = GETLOCAL(oparg);+ // do not use SETLOCAL here, it decrefs the old value+ GETLOCAL(oparg) = NULL;+ STACK_GROW(1);+ stack_pointer[-1] = value;+ break;+ }++ case LOAD_CONST: {+ PyObject *value;+ value = GETITEM(FRAME_CO_CONSTS, oparg);+ Py_INCREF(value);+ STACK_GROW(1);+ stack_pointer[-1] = value;+ break;+ }++ case STORE_FAST: {+ PyObject *value = stack_pointer[-1];+ SETLOCAL(oparg, value);+ STACK_SHRINK(1);+ break;+ }++ case POP_TOP: {+ PyObject *value = stack_pointer[-1];+ Py_DECREF(value);+ STACK_SHRINK(1);+ break;+ }++ case PUSH_NULL: {+ PyObject *res;+ res = NULL;+ STACK_GROW(1);+ stack_pointer[-1] = res;+ break;+ }++ case END_SEND: {+ PyObject *value = stack_pointer[-1];+ PyObject *receiver = stack_pointer[-2];+ Py_DECREF(receiver);+ STACK_SHRINK(1);+ stack_pointer[-1] = value;+ break;+ }++ case UNARY_NEGATIVE: {+ PyObject *value = stack_pointer[-1];+ PyObject *res;+ res = PyNumber_Negative(value);+ Py_DECREF(value);+ if (res == NULL) goto pop_1_error;+ stack_pointer[-1] = res;+ break;+ }++ case UNARY_NOT: {+ PyObject *value = stack_pointer[-1];+ PyObject *res;+ int err = PyObject_IsTrue(value);+ Py_DECREF(value);+ if (err < 0) goto pop_1_error;+ if (err == 0) {+ res = Py_True;+ }+ else {+ res = Py_False;+ }+ stack_pointer[-1] = res;+ break;+ }++ case UNARY_INVERT: {+ PyObject *value = stack_pointer[-1];+ PyObject *res;+ res = PyNumber_Invert(value);+ Py_DECREF(value);+ if (res == NULL) goto pop_1_error;+ stack_pointer[-1] = res;+ break;+ }++ case _GUARD_BOTH_INT: {+ PyObject *right = stack_pointer[-1];+ PyObject *left = stack_pointer[-2];+ DEOPT_IF(!PyLong_CheckExact(left), BINARY_OP);+ DEOPT_IF(!PyLong_CheckExact(right), BINARY_OP);+ break;+ }++ case _BINARY_OP_MULTIPLY_INT: {+ PyObject *right = stack_pointer[-1];+ PyObject *left = stack_pointer[-2];+ PyObject *res;+ STAT_INC(BINARY_OP, hit);+ res = _PyLong_Multiply((PyLongObject *)left, (PyLongObject *)right);+ _Py_DECREF_SPECIALIZED(right, (destructor)PyObject_Free);+ _Py_DECREF_SPECIALIZED(left, (destructor)PyObject_Free);+ if (res == NULL) goto pop_2_error;+ STACK_SHRINK(1);+ stack_pointer[-1] = res;+ break;+ }++ case _BINARY_OP_ADD_INT: {+ PyObject *right = stack_pointer[-1];+ PyObject *left = stack_pointer[-2];+ PyObject *res;+ STAT_INC(BINARY_OP, hit);+ res = _PyLong_Add((PyLongObject *)left, (PyLongObject *)right);+ _Py_DECREF_SPECIALIZED(right, (destructor)PyObject_Free);+ _Py_DECREF_SPECIALIZED(left, (destructor)PyObject_Free);+ if (res == NULL) goto pop_2_error;+ STACK_SHRINK(1);+ stack_pointer[-1] = res;+ break;+ }++ case _BINARY_OP_SUBTRACT_INT: {+ PyObject *right = stack_pointer[-1];+ PyObject *left = stack_pointer[-2];+ PyObject *res;+ STAT_INC(BINARY_OP, hit);+ res = _PyLong_Subtract((PyLongObject *)left, (PyLongObject *)right);+ _Py_DECREF_SPECIALIZED(right, (destructor)PyObject_Free);+ _Py_DECREF_SPECIALIZED(left, (destructor)PyObject_Free);+ if (res == NULL) goto pop_2_error;+ STACK_SHRINK(1);+ stack_pointer[-1] = res;+ break;+ }++ case _GUARD_BOTH_FLOAT: {+ PyObject *right = stack_pointer[-1];+ PyObject *left = stack_pointer[-2];+ DEOPT_IF(!PyFloat_CheckExact(left), BINARY_OP);+ DEOPT_IF(!PyFloat_CheckExact(right), BINARY_OP);+ break;+ }++ case _BINARY_OP_MULTIPLY_FLOAT: {+ PyObject *right = stack_pointer[-1];+ PyObject *left = stack_pointer[-2];+ PyObject *res;+ STAT_INC(BINARY_OP, hit);+ double dres =+ ((PyFloatObject *)left)->ob_fval *+ ((PyFloatObject *)right)->ob_fval;+ DECREF_INPUTS_AND_REUSE_FLOAT(left, right, dres, res);+ STACK_SHRINK(1);+ stack_pointer[-1] = res;+ break;+ }++ case _BINARY_OP_ADD_FLOAT: {+ PyObject *right = stack_pointer[-1];+ PyObject *left = stack_pointer[-2];+ PyObject *res;+ STAT_INC(BINARY_OP, hit);+ double dres =+ ((PyFloatObject *)left)->ob_fval ++ ((PyFloatObject *)right)->ob_fval;+ DECREF_INPUTS_AND_REUSE_FLOAT(left, right, dres, res);+ STACK_SHRINK(1);+ stack_pointer[-1] = res;+ break;+ }++ case _BINARY_OP_SUBTRACT_FLOAT: {+ PyObject *right = stack_pointer[-1];+ PyObject *left = stack_pointer[-2];+ PyObject *res;+ STAT_INC(BINARY_OP, hit);+ double dres =+ ((PyFloatObject *)left)->ob_fval -+ ((PyFloatObject *)right)->ob_fval;+ DECREF_INPUTS_AND_REUSE_FLOAT(left, right, dres, res);+ STACK_SHRINK(1);+ stack_pointer[-1] = res;+ break;+ }++ case _GUARD_BOTH_UNICODE: {+ PyObject *right = stack_pointer[-1];+ PyObject *left = stack_pointer[-2];+ DEOPT_IF(!PyUnicode_CheckExact(left), BINARY_OP);+ DEOPT_IF(!PyUnicode_CheckExact(right), BINARY_OP);+ break;+ }++ case _BINARY_OP_ADD_UNICODE: {+ PyObject *right = stack_pointer[-1];+ PyObject *left = stack_pointer[-2];+ PyObject *res;+ STAT_INC(BINARY_OP, hit);+ res = PyUnicode_Concat(left, right);+ _Py_DECREF_SPECIALIZED(left, _PyUnicode_ExactDealloc);+ _Py_DECREF_SPECIALIZED(right, _PyUnicode_ExactDealloc);+ if (res == NULL) goto pop_2_error;+ STACK_SHRINK(1);+ stack_pointer[-1] = res;+ break;+ }++ case BINARY_SLICE: {+ PyObject *stop = stack_pointer[-1];+ PyObject *start = stack_pointer[-2];+ PyObject *container = stack_pointer[-3];+ PyObject *res;+ PyObject *slice = _PyBuildSlice_ConsumeRefs(start, stop);+ // Can't use ERROR_IF() here, because we haven't+ // DECREF'ed container yet, and we still own slice.+ if (slice == NULL) {+ res = NULL;+ }+ else {+ res = PyObject_GetItem(container, slice);+ Py_DECREF(slice);+ }+ Py_DECREF(container);+ if (res == NULL) goto pop_3_error;+ STACK_SHRINK(2);+ stack_pointer[-1] = res;+ break;+ }++ case STORE_SLICE: {+ PyObject *stop = stack_pointer[-1];+ PyObject *start = stack_pointer[-2];+ PyObject *container = stack_pointer[-3];+ PyObject *v = stack_pointer[-4];+ PyObject *slice = _PyBuildSlice_ConsumeRefs(start, stop);+ int err;+ if (slice == NULL) {+ err = 1;+ }+ else {+ err = PyObject_SetItem(container, slice, v);+ Py_DECREF(slice);+ }+ Py_DECREF(v);+ Py_DECREF(container);+ if (err) goto pop_4_error;+ STACK_SHRINK(4);+ break;+ }++ case BINARY_SUBSCR_LIST_INT: {+ PyObject *sub = stack_pointer[-1];+ PyObject *list = stack_pointer[-2];+ PyObject *res;+ DEOPT_IF(!PyLong_CheckExact(sub), BINARY_SUBSCR);+ DEOPT_IF(!PyList_CheckExact(list), BINARY_SUBSCR);++ // Deopt unless 0 <= sub < PyList_Size(list)+ DEOPT_IF(!_PyLong_IsNonNegativeCompact((PyLongObject *)sub), BINARY_SUBSCR);+ Py_ssize_t index = ((PyLongObject*)sub)->long_value.ob_digit[0];+ DEOPT_IF(index >= PyList_GET_SIZE(list), BINARY_SUBSCR);+ STAT_INC(BINARY_SUBSCR, hit);+ res = PyList_GET_ITEM(list, index);+ assert(res != NULL);+ Py_INCREF(res);+ _Py_DECREF_SPECIALIZED(sub, (destructor)PyObject_Free);+ Py_DECREF(list);+ STACK_SHRINK(1);+ stack_pointer[-1] = res;+ break;+ }++ case BINARY_SUBSCR_TUPLE_INT: {+ PyObject *sub = stack_pointer[-1];+ PyObject *tuple = stack_pointer[-2];+ PyObject *res;+ DEOPT_IF(!PyLong_CheckExact(sub), BINARY_SUBSCR);+ DEOPT_IF(!PyTuple_CheckExact(tuple), BINARY_SUBSCR);++ // Deopt unless 0 <= sub < PyTuple_Size(list)+ DEOPT_IF(!_PyLong_IsNonNegativeCompact((PyLongObject *)sub), BINARY_SUBSCR);+ Py_ssize_t index = ((PyLongObject*)sub)->long_value.ob_digit[0];+ DEOPT_IF(index >= PyTuple_GET_SIZE(tuple), BINARY_SUBSCR);+ STAT_INC(BINARY_SUBSCR, hit);+ res = PyTuple_GET_ITEM(tuple, index);+ assert(res != NULL);+ Py_INCREF(res);+ _Py_DECREF_SPECIALIZED(sub, (destructor)PyObject_Free);+ Py_DECREF(tuple);+ STACK_SHRINK(1);+ stack_pointer[-1] = res;+ break;+ }++ case BINARY_SUBSCR_DICT: {+ PyObject *sub = stack_pointer[-1];+ PyObject *dict = stack_pointer[-2];+ PyObject *res;+ DEOPT_IF(!PyDict_CheckExact(dict), BINARY_SUBSCR);+ STAT_INC(BINARY_SUBSCR, hit);+ res = PyDict_GetItemWithError(dict, sub);+ if (res == NULL) {+ if (!_PyErr_Occurred(tstate)) {+ _PyErr_SetKeyError(sub);+ }+ Py_DECREF(dict);+ Py_DECREF(sub);+ if (true) goto pop_2_error;+ }+ Py_INCREF(res); // Do this before DECREF'ing dict, sub+ Py_DECREF(dict);+ Py_DECREF(sub);+ STACK_SHRINK(1);+ stack_pointer[-1] = res;+ break;+ }++ case LIST_APPEND: {+ PyObject *v = stack_pointer[-1];+ PyObject *list = stack_pointer[-(2 + (oparg-1))];+ if (_PyList_AppendTakeRef((PyListObject *)list, v) < 0) goto pop_1_error;+ STACK_SHRINK(1);+ break;+ }++ case SET_ADD: {+ PyObject *v = stack_pointer[-1];+ PyObject *set = stack_pointer[-(2 + (oparg-1))];+ int err = PySet_Add(set, v);+ Py_DECREF(v);+ if (err) goto pop_1_error;+ STACK_SHRINK(1);+ break;+ }++ case STORE_SUBSCR_LIST_INT: {+ PyObject *sub = stack_pointer[-1];+ PyObject *list = stack_pointer[-2];+ PyObject *value = stack_pointer[-3];+ DEOPT_IF(!PyLong_CheckExact(sub), STORE_SUBSCR);+ DEOPT_IF(!PyList_CheckExact(list), STORE_SUBSCR);++ // Ensure nonnegative, zero-or-one-digit ints.+ DEOPT_IF(!_PyLong_IsNonNegativeCompact((PyLongObject *)sub), STORE_SUBSCR);+ Py_ssize_t index = ((PyLongObject*)sub)->long_value.ob_digit[0];+ // Ensure index < len(list)+ DEOPT_IF(index >= PyList_GET_SIZE(list), STORE_SUBSCR);+ STAT_INC(STORE_SUBSCR, hit);++ PyObject *old_value = PyList_GET_ITEM(list, index);+ PyList_SET_ITEM(list, index, value);+ assert(old_value != NULL);+ Py_DECREF(old_value);+ _Py_DECREF_SPECIALIZED(sub, (destructor)PyObject_Free);+ Py_DECREF(list);+ STACK_SHRINK(3);+ break;+ }++ case STORE_SUBSCR_DICT: {+ PyObject *sub = stack_pointer[-1];+ PyObject *dict = stack_pointer[-2];+ PyObject *value = stack_pointer[-3];+ DEOPT_IF(!PyDict_CheckExact(dict), STORE_SUBSCR);+ STAT_INC(STORE_SUBSCR, hit);+ int err = _PyDict_SetItem_Take2((PyDictObject *)dict, sub, value);+ Py_DECREF(dict);+ if (err) goto pop_3_error;+ STACK_SHRINK(3);+ break;+ }++ case DELETE_SUBSCR: {+ PyObject *sub = stack_pointer[-1];+ PyObject *container = stack_pointer[-2];+ /* del container[sub] */+ int err = PyObject_DelItem(container, sub);+ Py_DECREF(container);+ Py_DECREF(sub);+ if (err) goto pop_2_error;+ STACK_SHRINK(2);+ break;+ }++ case CALL_INTRINSIC_1: {+ PyObject *value = stack_pointer[-1];+ PyObject *res;+ assert(oparg <= MAX_INTRINSIC_1);+ res = _PyIntrinsics_UnaryFunctions[oparg](tstate, value);+ Py_DECREF(value);+ if (res == NULL) goto pop_1_error;+ stack_pointer[-1] = res;+ break;+ }++ case CALL_INTRINSIC_2: {+ PyObject *value1 = stack_pointer[-1];+ PyObject *value2 = stack_pointer[-2];+ PyObject *res;+ assert(oparg <= MAX_INTRINSIC_2);+ res = _PyIntrinsics_BinaryFunctions[oparg](tstate, value2, value1);+ Py_DECREF(value2);+ Py_DECREF(value1);+ if (res == NULL) goto pop_2_error;+ STACK_SHRINK(1);+ stack_pointer[-1] = res;+ break;+ }++ case GET_AITER: {+ PyObject *obj = stack_pointer[-1];+ PyObject *iter;+ unaryfunc getter = NULL;+ PyTypeObject *type = Py_TYPE(obj);++ if (type->tp_as_async != NULL) {+ getter = type->tp_as_async->am_aiter;+ }++ if (getter == NULL) {+ _PyErr_Format(tstate, PyExc_TypeError,+ "'async for' requires an object with "+ "__aiter__ method, got %.100s",+ type->tp_name);+ Py_DECREF(obj);+ if (true) goto pop_1_error;+ }++ iter = (*getter)(obj);+ Py_DECREF(obj);+ if (iter == NULL) goto pop_1_error;++ if (Py_TYPE(iter)->tp_as_async == NULL ||+ Py_TYPE(iter)->tp_as_async->am_anext == NULL) {++ _PyErr_Format(tstate, PyExc_TypeError,+ "'async for' received an object from __aiter__ "+ "that does not implement __anext__: %.100s",+ Py_TYPE(iter)->tp_name);+ Py_DECREF(iter);+ if (true) goto pop_1_error;+ }+ stack_pointer[-1] = iter;+ break;+ }++ case GET_ANEXT: {+ PyObject *aiter = stack_pointer[-1];+ PyObject *awaitable;+ unaryfunc getter = NULL;+ PyObject *next_iter = NULL;+ PyTypeObject *type = Py_TYPE(aiter);++ if (PyAsyncGen_CheckExact(aiter)) {+ awaitable = type->tp_as_async->am_anext(aiter);+ if (awaitable == NULL) {+ goto error;+ }+ } else {+ if (type->tp_as_async != NULL){+ getter = type->tp_as_async->am_anext;+ }++ if (getter != NULL) {+ next_iter = (*getter)(aiter);+ if (next_iter == NULL) {+ goto error;+ }+ }+ else {+ _PyErr_Format(tstate, PyExc_TypeError,+ "'async for' requires an iterator with "+ "__anext__ method, got %.100s",+ type->tp_name);+ goto error;+ }++ awaitable = _PyCoro_GetAwaitableIter(next_iter);+ if (awaitable == NULL) {+ _PyErr_FormatFromCause(+ PyExc_TypeError,+ "'async for' received an invalid object "+ "from __anext__: %.100s",+ Py_TYPE(next_iter)->tp_name);++ Py_DECREF(next_iter);+ goto error;+ } else {+ Py_DECREF(next_iter);+ }+ }+ STACK_GROW(1);+ stack_pointer[-1] = awaitable;+ break;+ }++ case GET_AWAITABLE: {+ PyObject *iterable = stack_pointer[-1];+ PyObject *iter;+ iter = _PyCoro_GetAwaitableIter(iterable);++ if (iter == NULL) {+ format_awaitable_error(tstate, Py_TYPE(iterable), oparg);+ }++ Py_DECREF(iterable);++ if (iter != NULL && PyCoro_CheckExact(iter)) {+ PyObject *yf = _PyGen_yf((PyGenObject*)iter);+ if (yf != NULL) {+ /* `iter` is a coroutine object that is being+ awaited, `yf` is a pointer to the current awaitable+ being awaited on. */+ Py_DECREF(yf);+ Py_CLEAR(iter);+ _PyErr_SetString(tstate, PyExc_RuntimeError,+ "coroutine is being awaited already");+ /* The code below jumps to `error` if `iter` is NULL. */+ }+ }++ if (iter == NULL) goto pop_1_error;+ stack_pointer[-1] = iter;+ break;+ }++ case POP_EXCEPT: {+ PyObject *exc_value = stack_pointer[-1];+ _PyErr_StackItem *exc_info = tstate->exc_info;+ Py_XSETREF(exc_info->exc_value, exc_value);+ STACK_SHRINK(1);+ break;+ }++ case LOAD_ASSERTION_ERROR: {+ PyObject *value;+ value = Py_NewRef(PyExc_AssertionError);+ STACK_GROW(1);+ stack_pointer[-1] = value;+ break;+ }++ case LOAD_BUILD_CLASS: {+ PyObject *bc;+ if (PyDict_CheckExact(BUILTINS())) {+ bc = _PyDict_GetItemWithError(BUILTINS(),+ &_Py_ID(__build_class__));+ if (bc == NULL) {+ if (!_PyErr_Occurred(tstate)) {+ _PyErr_SetString(tstate, PyExc_NameError,+ "__build_class__ not found");+ }+ if (true) goto error;+ }+ Py_INCREF(bc);+ }+ else {+ bc = PyObject_GetItem(BUILTINS(), &_Py_ID(__build_class__));+ if (bc == NULL) {+ if (_PyErr_ExceptionMatches(tstate, PyExc_KeyError))+ _PyErr_SetString(tstate, PyExc_NameError,+ "__build_class__ not found");+ if (true) goto error;+ }+ }+ STACK_GROW(1);+ stack_pointer[-1] = bc;+ break;+ }++ case STORE_NAME: {+ PyObject *v = stack_pointer[-1];+ PyObject *name = GETITEM(FRAME_CO_NAMES, oparg);+ PyObject *ns = LOCALS();+ int err;+ if (ns == NULL) {+ _PyErr_Format(tstate, PyExc_SystemError,+ "no locals found when storing %R", name);+ Py_DECREF(v);+ if (true) goto pop_1_error;+ }+ if (PyDict_CheckExact(ns))+ err = PyDict_SetItem(ns, name, v);+ else+ err = PyObject_SetItem(ns, name, v);+ Py_DECREF(v);+ if (err) goto pop_1_error;+ STACK_SHRINK(1);+ break;+ }++ case DELETE_NAME: {+ PyObject *name = GETITEM(FRAME_CO_NAMES, oparg);+ PyObject *ns = LOCALS();+ int err;+ if (ns == NULL) {+ _PyErr_Format(tstate, PyExc_SystemError,+ "no locals when deleting %R", name);+ goto error;+ }+ err = PyObject_DelItem(ns, name);+ // Can't use ERROR_IF here.+ if (err != 0) {+ format_exc_check_arg(tstate, PyExc_NameError,+ NAME_ERROR_MSG,+ name);+ goto error;+ }+ break;+ }++ case UNPACK_SEQUENCE_TWO_TUPLE: {+ PyObject *seq = stack_pointer[-1];+ PyObject **values = stack_pointer - (1);+ DEOPT_IF(!PyTuple_CheckExact(seq), UNPACK_SEQUENCE);+ DEOPT_IF(PyTuple_GET_SIZE(seq) != 2, UNPACK_SEQUENCE);+ assert(oparg == 2);+ STAT_INC(UNPACK_SEQUENCE, hit);+ values[0] = Py_NewRef(PyTuple_GET_ITEM(seq, 1));+ values[1] = Py_NewRef(PyTuple_GET_ITEM(seq, 0));+ Py_DECREF(seq);+ STACK_SHRINK(1);+ STACK_GROW(oparg);+ break;+ }++ case UNPACK_SEQUENCE_TUPLE: {+ PyObject *seq = stack_pointer[-1];+ PyObject **values = stack_pointer - (1);+ DEOPT_IF(!PyTuple_CheckExact(seq), UNPACK_SEQUENCE);+ DEOPT_IF(PyTuple_GET_SIZE(seq) != oparg, UNPACK_SEQUENCE);+ STAT_INC(UNPACK_SEQUENCE, hit);+ PyObject **items = _PyTuple_ITEMS(seq);+ for (int i = oparg; --i >= 0; ) {+ *values++ = Py_NewRef(items[i]);+ }+ Py_DECREF(seq);+ STACK_SHRINK(1);+ STACK_GROW(oparg);+ break;+ }++ case UNPACK_SEQUENCE_LIST: {+ PyObject *seq = stack_pointer[-1];+ PyObject **values = stack_pointer - (1);+ DEOPT_IF(!PyList_CheckExact(seq), UNPACK_SEQUENCE);+ DEOPT_IF(PyList_GET_SIZE(seq) != oparg, UNPACK_SEQUENCE);+ STAT_INC(UNPACK_SEQUENCE, hit);+ PyObject **items = _PyList_ITEMS(seq);+ for (int i = oparg; --i >= 0; ) {+ *values++ = Py_NewRef(items[i]);+ }+ Py_DECREF(seq);+ STACK_SHRINK(1);+ STACK_GROW(oparg);+ break;+ }++ case UNPACK_EX: {+ PyObject *seq = stack_pointer[-1];+ int totalargs = 1 + (oparg & 0xFF) + (oparg >> 8);+ PyObject **top = stack_pointer + totalargs - 1;+ int res = unpack_iterable(tstate, seq, oparg & 0xFF, oparg >> 8, top);+ Py_DECREF(seq);+ if (res == 0) goto pop_1_error;+ STACK_GROW((oparg & 0xFF) + (oparg >> 8));+ break;+ }++ case DELETE_ATTR: {+ PyObject *owner = stack_pointer[-1];+ PyObject *name = GETITEM(FRAME_CO_NAMES, oparg);+ int err = PyObject_SetAttr(owner, name, (PyObject *)NULL);+ Py_DECREF(owner);+ if (err) goto pop_1_error;+ STACK_SHRINK(1);+ break;+ }++ case STORE_GLOBAL: {+ PyObject *v = stack_pointer[-1];+ PyObject *name = GETITEM(FRAME_CO_NAMES, oparg);+ int err = PyDict_SetItem(GLOBALS(), name, v);+ Py_DECREF(v);+ if (err) goto pop_1_error;+ STACK_SHRINK(1);+ break;+ }++ case DELETE_GLOBAL: {+ PyObject *name = GETITEM(FRAME_CO_NAMES, oparg);+ int err;+ err = PyDict_DelItem(GLOBALS(), name);+ // Can't use ERROR_IF here.+ if (err != 0) {+ if (_PyErr_ExceptionMatches(tstate, PyExc_KeyError)) {+ format_exc_check_arg(tstate, PyExc_NameError,+ NAME_ERROR_MSG, name);+ }+ goto error;+ }+ break;+ }++ case _LOAD_LOCALS: {+ PyObject *locals;+ locals = LOCALS();+ if (locals == NULL) {+ _PyErr_SetString(tstate, PyExc_SystemError,+ "no locals found");+ if (true) goto error;+ }+ Py_INCREF(locals);+ STACK_GROW(1);+ stack_pointer[-1] = locals;+ break;+ }++ case _LOAD_FROM_DICT_OR_GLOBALS: {+ PyObject *mod_or_class_dict = stack_pointer[-1];+ PyObject *v;+ PyObject *name = GETITEM(FRAME_CO_NAMES, oparg);+ if (PyDict_CheckExact(mod_or_class_dict)) {+ v = PyDict_GetItemWithError(mod_or_class_dict, name);+ if (v != NULL) {+ Py_INCREF(v);+ }+ else if (_PyErr_Occurred(tstate)) {+ Py_DECREF(mod_or_class_dict);+ goto error;+ }+ }+ else {+ v = PyObject_GetItem(mod_or_class_dict, name);+ if (v == NULL) {+ if (!_PyErr_ExceptionMatches(tstate, PyExc_KeyError)) {+ Py_DECREF(mod_or_class_dict);+ goto error;+ }+ _PyErr_Clear(tstate);+ }+ }+ Py_DECREF(mod_or_class_dict);+ if (v == NULL) {+ v = PyDict_GetItemWithError(GLOBALS(), name);+ if (v != NULL) {+ Py_INCREF(v);+ }+ else if (_PyErr_Occurred(tstate)) {+ goto error;+ }+ else {+ if (PyDict_CheckExact(BUILTINS())) {+ v = PyDict_GetItemWithError(BUILTINS(), name);+ if (v == NULL) {+ if (!_PyErr_Occurred(tstate)) {+ format_exc_check_arg(+ tstate, PyExc_NameError,+ NAME_ERROR_MSG, name);+ }+ goto error;+ }+ Py_INCREF(v);+ }+ else {+ v = PyObject_GetItem(BUILTINS(), name);+ if (v == NULL) {+ if (_PyErr_ExceptionMatches(tstate, PyExc_KeyError)) {+ format_exc_check_arg(+ tstate, PyExc_NameError,+ NAME_ERROR_MSG, name);+ }+ goto error;+ }+ }+ }+ }+ stack_pointer[-1] = v;+ break;+ }++ case DELETE_DEREF: {+ PyObject *cell = GETLOCAL(oparg);+ PyObject *oldobj = PyCell_GET(cell);+ // Can't use ERROR_IF here.+ // Fortunately we don't need its superpower.+ if (oldobj == NULL) {+ format_exc_unbound(tstate, _PyFrame_GetCode(frame), oparg);+ goto error;+ }+ PyCell_SET(cell, NULL);+ Py_DECREF(oldobj);+ break;+ }++ case LOAD_FROM_DICT_OR_DEREF: {+ PyObject *class_dict = stack_pointer[-1];+ PyObject *value;+ PyObject *name;+ assert(class_dict);+ assert(oparg >= 0 && oparg < _PyFrame_GetCode(frame)->co_nlocalsplus);+ name = PyTuple_GET_ITEM(_PyFrame_GetCode(frame)->co_localsplusnames, oparg);+ if (PyDict_CheckExact(class_dict)) {+ value = PyDict_GetItemWithError(class_dict, name);+ if (value != NULL) {+ Py_INCREF(value);+ }+ else if (_PyErr_Occurred(tstate)) {+ Py_DECREF(class_dict);+ goto error;+ }+ }+ else {+ value = PyObject_GetItem(class_dict, name);+ if (value == NULL) {+ if (!_PyErr_ExceptionMatches(tstate, PyExc_KeyError)) {+ Py_DECREF(class_dict);+ goto error;+ }+ _PyErr_Clear(tstate);+ }+ }+ Py_DECREF(class_dict);+ if (!value) {+ PyObject *cell = GETLOCAL(oparg);+ value = PyCell_GET(cell);+ if (value == NULL) {+ format_exc_unbound(tstate, _PyFrame_GetCode(frame), oparg);+ goto error;+ }+ Py_INCREF(value);+ }+ stack_pointer[-1] = value;+ break;+ }++ case LOAD_DEREF: {+ PyObject *value;+ PyObject *cell = GETLOCAL(oparg);+ value = PyCell_GET(cell);+ if (value == NULL) {+ format_exc_unbound(tstate, _PyFrame_GetCode(frame), oparg);+ if (true) goto error;+ }+ Py_INCREF(value);+ STACK_GROW(1);+ stack_pointer[-1] = value;+ break;+ }++ case STORE_DEREF: {+ PyObject *v = stack_pointer[-1];+ PyObject *cell = GETLOCAL(oparg);+ PyObject *oldobj = PyCell_GET(cell);+ PyCell_SET(cell, v);+ Py_XDECREF(oldobj);+ STACK_SHRINK(1);+ break;+ }++ case COPY_FREE_VARS: {+ /* Copy closure variables to free variables */+ PyCodeObject *co = _PyFrame_GetCode(frame);+ assert(PyFunction_Check(frame->f_funcobj));+ PyObject *closure = ((PyFunctionObject *)frame->f_funcobj)->func_closure;+ assert(oparg == co->co_nfreevars);+ int offset = co->co_nlocalsplus - oparg;+ for (int i = 0; i < oparg; ++i) {+ PyObject *o = PyTuple_GET_ITEM(closure, i);+ frame->localsplus[offset + i] = Py_NewRef(o);+ }+ break;+ }++ case BUILD_STRING: {+ PyObject **pieces = (stack_pointer - oparg);+ PyObject *str;+ str = _PyUnicode_JoinArray(&_Py_STR(empty), pieces, oparg);+ for (int _i = oparg; --_i >= 0;) {+ Py_DECREF(pieces[_i]);+ }+ if (str == NULL) { STACK_SHRINK(oparg); goto error; }+ STACK_SHRINK(oparg);+ STACK_GROW(1);+ stack_pointer[-1] = str;+ break;+ }++ case BUILD_TUPLE: {+ PyObject **values = (stack_pointer - oparg);+ PyObject *tup;+ tup = _PyTuple_FromArraySteal(values, oparg);+ if (tup == NULL) { STACK_SHRINK(oparg); goto error; }+ STACK_SHRINK(oparg);+ STACK_GROW(1);+ stack_pointer[-1] = tup;+ break;+ }++ case BUILD_LIST: {+ PyObject **values = (stack_pointer - oparg);+ PyObject *list;+ list = _PyList_FromArraySteal(values, oparg);+ if (list == NULL) { STACK_SHRINK(oparg); goto error; }+ STACK_SHRINK(oparg);+ STACK_GROW(1);+ stack_pointer[-1] = list;+ break;+ }++ case LIST_EXTEND: {+ PyObject *iterable = stack_pointer[-1];+ PyObject *list = stack_pointer[-(2 + (oparg-1))];+ PyObject *none_val = _PyList_Extend((PyListObject *)list, iterable);+ if (none_val == NULL) {+ if (_PyErr_ExceptionMatches(tstate, PyExc_TypeError) &&+ (Py_TYPE(iterable)->tp_iter == NULL && !PySequence_Check(iterable)))+ {+ _PyErr_Clear(tstate);+ _PyErr_Format(tstate, PyExc_TypeError,+ "Value after * must be an iterable, not %.200s",+ Py_TYPE(iterable)->tp_name);+ }+ Py_DECREF(iterable);+ if (true) goto pop_1_error;+ }+ assert(Py_IsNone(none_val));+ Py_DECREF(iterable);+ STACK_SHRINK(1);+ break;+ }++ case SET_UPDATE: {+ PyObject *iterable = stack_pointer[-1];+ PyObject *set = stack_pointer[-(2 + (oparg-1))];+ int err = _PySet_Update(set, iterable);+ Py_DECREF(iterable);+ if (err < 0) goto pop_1_error;+ STACK_SHRINK(1);+ break;+ }++ case BUILD_SET: {+ PyObject **values = (stack_pointer - oparg);+ PyObject *set;+ set = PySet_New(NULL);+ if (set == NULL)+ goto error;+ int err = 0;+ for (int i = 0; i < oparg; i++) {+ PyObject *item = values[i];+ if (err == 0)+ err = PySet_Add(set, item);+ Py_DECREF(item);+ }+ if (err != 0) {+ Py_DECREF(set);+ if (true) { STACK_SHRINK(oparg); goto error; }+ }+ STACK_SHRINK(oparg);+ STACK_GROW(1);+ stack_pointer[-1] = set;+ break;+ }++ case BUILD_MAP: {+ PyObject **values = (stack_pointer - oparg*2);+ PyObject *map;+ map = _PyDict_FromItems(+ values, 2,+ values+1, 2,+ oparg);+ if (map == NULL)+ goto error;++ for (int _i = oparg*2; --_i >= 0;) {+ Py_DECREF(values[_i]);+ }+ if (map == NULL) { STACK_SHRINK(oparg*2); goto error; }+ STACK_SHRINK(oparg*2);+ STACK_GROW(1);+ stack_pointer[-1] = map;+ break;+ }++ case SETUP_ANNOTATIONS: {+ int err;+ PyObject *ann_dict;+ if (LOCALS() == NULL) {+ _PyErr_Format(tstate, PyExc_SystemError,+ "no locals found when setting up annotations");+ if (true) goto error;+ }+ /* check if __annotations__ in locals()... */+ if (PyDict_CheckExact(LOCALS())) {+ ann_dict = _PyDict_GetItemWithError(LOCALS(),+ &_Py_ID(__annotations__));+ if (ann_dict == NULL) {+ if (_PyErr_Occurred(tstate)) goto error;+ /* ...if not, create a new one */+ ann_dict = PyDict_New();+ if (ann_dict == NULL) goto error;+ err = PyDict_SetItem(LOCALS(), &_Py_ID(__annotations__),+ ann_dict);+ Py_DECREF(ann_dict);+ if (err) goto error;+ }+ }+ else {+ /* do the same if locals() is not a dict */+ ann_dict = PyObject_GetItem(LOCALS(), &_Py_ID(__annotations__));+ if (ann_dict == NULL) {+ if (!_PyErr_ExceptionMatches(tstate, PyExc_KeyError)) goto error;+ _PyErr_Clear(tstate);+ ann_dict = PyDict_New();+ if (ann_dict == NULL) goto error;+ err = PyObject_SetItem(LOCALS(), &_Py_ID(__annotations__),+ ann_dict);+ Py_DECREF(ann_dict);+ if (err) goto error;+ }+ else {+ Py_DECREF(ann_dict);+ }+ }+ break;+ }++ case BUILD_CONST_KEY_MAP: {+ PyObject *keys = stack_pointer[-1];+ PyObject **values = (stack_pointer - (1 + oparg));+ PyObject *map;+ if (!PyTuple_CheckExact(keys) ||+ PyTuple_GET_SIZE(keys) != (Py_ssize_t)oparg) {+ _PyErr_SetString(tstate, PyExc_SystemError,+ "bad BUILD_CONST_KEY_MAP keys argument");+ goto error; // Pop the keys and values.+ }+ map = _PyDict_FromItems(+ &PyTuple_GET_ITEM(keys, 0), 1,+ values, 1, oparg);+ for (int _i = oparg; --_i >= 0;) {+ Py_DECREF(values[_i]);+ }+ Py_DECREF(keys);+ if (map == NULL) { STACK_SHRINK(oparg); goto pop_1_error; }+ STACK_SHRINK(oparg);+ stack_pointer[-1] = map;+ break;+ }++ case DICT_UPDATE: {+ PyObject *update = stack_pointer[-1];+ PyObject *dict = PEEK(oparg + 1); // update is still on the stack+ if (PyDict_Update(dict, update) < 0) {+ if (_PyErr_ExceptionMatches(tstate, PyExc_AttributeError)) {+ _PyErr_Format(tstate, PyExc_TypeError,+ "'%.200s' object is not a mapping",+ Py_TYPE(update)->tp_name);+ }+ Py_DECREF(update);+ if (true) goto pop_1_error;+ }+ Py_DECREF(update);+ STACK_SHRINK(1);+ break;+ }++ case DICT_MERGE: {+ PyObject *update = stack_pointer[-1];+ PyObject *dict = PEEK(oparg + 1); // update is still on the stack++ if (_PyDict_MergeEx(dict, update, 2) < 0) {+ format_kwargs_error(tstate, PEEK(3 + oparg), update);+ Py_DECREF(update);+ if (true) goto pop_1_error;+ }+ Py_DECREF(update);+ STACK_SHRINK(1);+ break;+ }++ case MAP_ADD: {+ PyObject *value = stack_pointer[-1];+ PyObject *key = stack_pointer[-2];+ PyObject *dict = PEEK(oparg + 2); // key, value are still on the stack+ assert(PyDict_CheckExact(dict));+ /* dict[key] = value */+ // Do not DECREF INPUTS because the function steals the references+ if (_PyDict_SetItem_Take2((PyDictObject *)dict, key, value) != 0) goto pop_2_error;+ STACK_SHRINK(2);+ break;+ }++ case LOAD_SUPER_ATTR_ATTR: {+ PyObject *self = stack_pointer[-1];+ PyObject *class = stack_pointer[-2];+ PyObject *global_super = stack_pointer[-3];+ PyObject *res2 = NULL;+ PyObject *res;+ assert(!(oparg & 1));+ DEOPT_IF(global_super != (PyObject *)&PySuper_Type, LOAD_SUPER_ATTR);+ DEOPT_IF(!PyType_Check(class), LOAD_SUPER_ATTR);+ STAT_INC(LOAD_SUPER_ATTR, hit);+ PyObject *name = GETITEM(FRAME_CO_NAMES, oparg >> 2);+ res = _PySuper_Lookup((PyTypeObject *)class, self, name, NULL);+ Py_DECREF(global_super);+ Py_DECREF(class);+ Py_DECREF(self);+ if (res == NULL) goto pop_3_error;+ STACK_SHRINK(2);+ STACK_GROW(((oparg & 1) ? 1 : 0));+ stack_pointer[-1] = res;+ if (oparg & 1) { stack_pointer[-(1 + ((oparg & 1) ? 1 : 0))] = res2; }+ break;+ }++ case LOAD_SUPER_ATTR_METHOD: {+ PyObject *self = stack_pointer[-1];+ PyObject *class = stack_pointer[-2];+ PyObject *global_super = stack_pointer[-3];+ PyObject *res2;+ PyObject *res;+ assert(oparg & 1);+ DEOPT_IF(global_super != (PyObject *)&PySuper_Type, LOAD_SUPER_ATTR);+ DEOPT_IF(!PyType_Check(class), LOAD_SUPER_ATTR);+ STAT_INC(LOAD_SUPER_ATTR, hit);+ PyObject *name = GETITEM(FRAME_CO_NAMES, oparg >> 2);+ PyTypeObject *cls = (PyTypeObject *)class;+ int method_found = 0;+ res2 = _PySuper_Lookup(cls, self, name,+ cls->tp_getattro == PyObject_GenericGetAttr ? &method_found : NULL);+ Py_DECREF(global_super);+ Py_DECREF(class);+ if (res2 == NULL) {+ Py_DECREF(self);+ if (true) goto pop_3_error;+ }+ if (method_found) {+ res = self; // transfer ownership+ } else {+ Py_DECREF(self);+ res = res2;+ res2 = NULL;+ }+ STACK_SHRINK(1);+ stack_pointer[-1] = res;+ stack_pointer[-2] = res2;+ break;+ }++ case COMPARE_OP_FLOAT: {+ PyObject *right = stack_pointer[-1];+ PyObject *left = stack_pointer[-2];+ PyObject *res;+ DEOPT_IF(!PyFloat_CheckExact(left), COMPARE_OP);+ DEOPT_IF(!PyFloat_CheckExact(right), COMPARE_OP);+ STAT_INC(COMPARE_OP, hit);+ double dleft = PyFloat_AS_DOUBLE(left);+ double dright = PyFloat_AS_DOUBLE(right);+ // 1 if NaN, 2 if <, 4 if >, 8 if ==; this matches low four bits of the oparg+ int sign_ish = COMPARISON_BIT(dleft, dright);+ _Py_DECREF_SPECIALIZED(left, _PyFloat_ExactDealloc);+ _Py_DECREF_SPECIALIZED(right, _PyFloat_ExactDealloc);+ res = (sign_ish & oparg) ? Py_True : Py_False;+ STACK_SHRINK(1);+ stack_pointer[-1] = res;+ break;+ }++ case COMPARE_OP_INT: {+ PyObject *right = stack_pointer[-1];+ PyObject *left = stack_pointer[-2];+ PyObject *res;+ DEOPT_IF(!PyLong_CheckExact(left), COMPARE_OP);+ DEOPT_IF(!PyLong_CheckExact(right), COMPARE_OP);+ DEOPT_IF(!_PyLong_IsCompact((PyLongObject *)left), COMPARE_OP);+ DEOPT_IF(!_PyLong_IsCompact((PyLongObject *)right), COMPARE_OP);+ STAT_INC(COMPARE_OP, hit);+ assert(_PyLong_DigitCount((PyLongObject *)left) <= 1 &&+ _PyLong_DigitCount((PyLongObject *)right) <= 1);+ Py_ssize_t ileft = _PyLong_CompactValue((PyLongObject *)left);+ Py_ssize_t iright = _PyLong_CompactValue((PyLongObject *)right);+ // 2 if <, 4 if >, 8 if ==; this matches the low 4 bits of the oparg+ int sign_ish = COMPARISON_BIT(ileft, iright);+ _Py_DECREF_SPECIALIZED(left, (destructor)PyObject_Free);+ _Py_DECREF_SPECIALIZED(right, (destructor)PyObject_Free);+ res = (sign_ish & oparg) ? Py_True : Py_False;+ STACK_SHRINK(1);+ stack_pointer[-1] = res;+ break;+ }++ case COMPARE_OP_STR: {+ PyObject *right = stack_pointer[-1];+ PyObject *left = stack_pointer[-2];+ PyObject *res;+ DEOPT_IF(!PyUnicode_CheckExact(left), COMPARE_OP);+ DEOPT_IF(!PyUnicode_CheckExact(right), COMPARE_OP);+ STAT_INC(COMPARE_OP, hit);+ int eq = _PyUnicode_Equal(left, right);+ assert((oparg >>4) == Py_EQ || (oparg >>4) == Py_NE);+ _Py_DECREF_SPECIALIZED(left, _PyUnicode_ExactDealloc);+ _Py_DECREF_SPECIALIZED(right, _PyUnicode_ExactDealloc);+ assert(eq == 0 || eq == 1);+ assert((oparg & 0xf) == COMPARISON_NOT_EQUALS || (oparg & 0xf) == COMPARISON_EQUALS);+ assert(COMPARISON_NOT_EQUALS + 1 == COMPARISON_EQUALS);+ res = ((COMPARISON_NOT_EQUALS + eq) & oparg) ? Py_True : Py_False;+ STACK_SHRINK(1);+ stack_pointer[-1] = res;+ break;+ }++ case IS_OP: {+ PyObject *right = stack_pointer[-1];+ PyObject *left = stack_pointer[-2];+ PyObject *b;+ int res = Py_Is(left, right) ^ oparg;+ Py_DECREF(left);+ Py_DECREF(right);+ b = res ? Py_True : Py_False;+ STACK_SHRINK(1);+ stack_pointer[-1] = b;+ break;+ }++ case CONTAINS_OP: {+ PyObject *right = stack_pointer[-1];+ PyObject *left = stack_pointer[-2];+ PyObject *b;+ int res = PySequence_Contains(right, left);+ Py_DECREF(left);+ Py_DECREF(right);+ if (res < 0) goto pop_2_error;+ b = (res ^ oparg) ? Py_True : Py_False;+ STACK_SHRINK(1);+ stack_pointer[-1] = b;+ break;+ }++ case CHECK_EG_MATCH: {+ PyObject *match_type = stack_pointer[-1];+ PyObject *exc_value = stack_pointer[-2];+ PyObject *rest;+ PyObject *match;+ if (check_except_star_type_valid(tstate, match_type) < 0) {+ Py_DECREF(exc_value);+ Py_DECREF(match_type);+ if (true) goto pop_2_error;+ }++ match = NULL;+ rest = NULL;+ int res = exception_group_match(exc_value, match_type,+ &match, &rest);+ Py_DECREF(exc_value);+ Py_DECREF(match_type);+ if (res < 0) goto pop_2_error;++ assert((match == NULL) == (rest == NULL));+ if (match == NULL) goto pop_2_error;++ if (!Py_IsNone(match)) {+ PyErr_SetHandledException(match);+ }+ stack_pointer[-1] = match;+ stack_pointer[-2] = rest;+ break;+ }++ case CHECK_EXC_MATCH: {+ PyObject *right = stack_pointer[-1];+ PyObject *left = stack_pointer[-2];+ PyObject *b;+ assert(PyExceptionInstance_Check(left));+ if (check_except_type_valid(tstate, right) < 0) {+ Py_DECREF(right);+ if (true) goto pop_1_error;+ }++ int res = PyErr_GivenExceptionMatches(left, right);+ Py_DECREF(right);+ b = res ? Py_True : Py_False;+ stack_pointer[-1] = b;+ break;+ }++ case GET_LEN: {+ PyObject *obj = stack_pointer[-1];+ PyObject *len_o;+ // PUSH(len(TOS))+ Py_ssize_t len_i = PyObject_Length(obj);+ if (len_i < 0) goto error;+ len_o = PyLong_FromSsize_t(len_i);+ if (len_o == NULL) goto error;+ STACK_GROW(1);+ stack_pointer[-1] = len_o;+ break;+ }++ case MATCH_CLASS: {+ PyObject *names = stack_pointer[-1];+ PyObject *type = stack_pointer[-2];+ PyObject *subject = stack_pointer[-3];+ PyObject *attrs;+ // Pop TOS and TOS1. Set TOS to a tuple of attributes on success, or+ // None on failure.+ assert(PyTuple_CheckExact(names));+ attrs = match_class(tstate, subject, type, oparg, names);+ Py_DECREF(subject);+ Py_DECREF(type);+ Py_DECREF(names);+ if (attrs) {+ assert(PyTuple_CheckExact(attrs)); // Success!+ }+ else {+ if (_PyErr_Occurred(tstate)) goto pop_3_error;+ attrs = Py_None; // Failure!+ }+ STACK_SHRINK(2);+ stack_pointer[-1] = attrs;+ break;+ }++ case MATCH_MAPPING: {+ PyObject *subject = stack_pointer[-1];+ PyObject *res;+ int match = Py_TYPE(subject)->tp_flags & Py_TPFLAGS_MAPPING;+ res = match ? Py_True : Py_False;+ STACK_GROW(1);+ stack_pointer[-1] = res;+ break;+ }++ case MATCH_SEQUENCE: {+ PyObject *subject = stack_pointer[-1];+ PyObject *res;+ int match = Py_TYPE(subject)->tp_flags & Py_TPFLAGS_SEQUENCE;+ res = match ? Py_True : Py_False;+ STACK_GROW(1);+ stack_pointer[-1] = res;+ break;+ }++ case MATCH_KEYS: {+ PyObject *keys = stack_pointer[-1];+ PyObject *subject = stack_pointer[-2];+ PyObject *values_or_none;+ // On successful match, PUSH(values). Otherwise, PUSH(None).+ values_or_none = match_keys(tstate, subject, keys);+ if (values_or_none == NULL) goto error;+ STACK_GROW(1);+ stack_pointer[-1] = values_or_none;+ break;+ }++ case GET_ITER: {+ PyObject *iterable = stack_pointer[-1];+ PyObject *iter;+ /* before: [obj]; after [getiter(obj)] */+ iter = PyObject_GetIter(iterable);+ Py_DECREF(iterable);+ if (iter == NULL) goto pop_1_error;+ stack_pointer[-1] = iter;+ break;+ }++ case GET_YIELD_FROM_ITER: {+ PyObject *iterable = stack_pointer[-1];+ PyObject *iter;+ /* before: [obj]; after [getiter(obj)] */+ if (PyCoro_CheckExact(iterable)) {+ /* `iterable` is a coroutine */+ if (!(_PyFrame_GetCode(frame)->co_flags & (CO_COROUTINE | CO_ITERABLE_COROUTINE))) {+ /* and it is used in a 'yield from' expression of a+ regular generator. */+ _PyErr_SetString(tstate, PyExc_TypeError,+ "cannot 'yield from' a coroutine object "+ "in a non-coroutine generator");+ goto error;+ }+ iter = iterable;+ }+ else if (PyGen_CheckExact(iterable)) {+ iter = iterable;+ }+ else {+ /* `iterable` is not a generator. */+ iter = PyObject_GetIter(iterable);+ if (iter == NULL) {+ goto error;+ }+ Py_DECREF(iterable);+ }+ stack_pointer[-1] = iter;+ break;+ }++ case WITH_EXCEPT_START: {+ PyObject *val = stack_pointer[-1];+ PyObject *lasti = stack_pointer[-3];+ PyObject *exit_func = stack_pointer[-4];+ PyObject *res;+ /* At the top of the stack are 4 values:+ - val: TOP = exc_info()+ - unused: SECOND = previous exception+ - lasti: THIRD = lasti of exception in exc_info()+ - exit_func: FOURTH = the context.__exit__ bound method+ We call FOURTH(type(TOP), TOP, GetTraceback(TOP)).+ Then we push the __exit__ return value.+ */+ PyObject *exc, *tb;++ assert(val && PyExceptionInstance_Check(val));+ exc = PyExceptionInstance_Class(val);+ tb = PyException_GetTraceback(val);+ Py_XDECREF(tb);+ assert(PyLong_Check(lasti));+ (void)lasti; // Shut up compiler warning if asserts are off+ PyObject *stack[4] = {NULL, exc, val, tb};+ res = PyObject_Vectorcall(exit_func, stack + 1,+ 3 | PY_VECTORCALL_ARGUMENTS_OFFSET, NULL);+ if (res == NULL) goto error;+ STACK_GROW(1);+ stack_pointer[-1] = res;+ break;+ }++ case PUSH_EXC_INFO: {+ PyObject *new_exc = stack_pointer[-1];+ PyObject *prev_exc;+ _PyErr_StackItem *exc_info = tstate->exc_info;+ if (exc_info->exc_value != NULL) {+ prev_exc = exc_info->exc_value;+ }+ else {+ prev_exc = Py_None;+ }+ assert(PyExceptionInstance_Check(new_exc));+ exc_info->exc_value = Py_NewRef(new_exc);+ STACK_GROW(1);+ stack_pointer[-1] = new_exc;+ stack_pointer[-2] = prev_exc;+ break;+ }++ case EXIT_INIT_CHECK: {+ PyObject *should_be_none = stack_pointer[-1];+ assert(STACK_LEVEL() == 2);+ if (should_be_none != Py_None) {+ PyErr_Format(PyExc_TypeError,+ "__init__() should return None, not '%.200s'",+ Py_TYPE(should_be_none)->tp_name);+ goto error;+ }+ STACK_SHRINK(1);+ break;+ }++ case MAKE_FUNCTION: {+ PyObject *codeobj = stack_pointer[-1];+ PyObject *func;++ PyFunctionObject *func_obj = (PyFunctionObject *)+ PyFunction_New(codeobj, GLOBALS());++ Py_DECREF(codeobj);+ if (func_obj == NULL) {+ goto error;+ }++ func_obj->func_version = ((PyCodeObject *)codeobj)->co_version;+ func = (PyObject *)func_obj;+ stack_pointer[-1] = func;+ break;+ }++ case SET_FUNCTION_ATTRIBUTE: {+ PyObject *func = stack_pointer[-1];+ PyObject *attr = stack_pointer[-2];+ assert(PyFunction_Check(func));+ PyFunctionObject *func_obj = (PyFunctionObject *)func;+ switch(oparg) {+ case MAKE_FUNCTION_CLOSURE:+ assert(func_obj->func_closure == NULL);+ func_obj->func_closure = attr;+ break;+ case MAKE_FUNCTION_ANNOTATIONS:+ assert(func_obj->func_annotations == NULL);+ func_obj->func_annotations = attr;+ break;+ case MAKE_FUNCTION_KWDEFAULTS:+ assert(PyDict_CheckExact(attr));+ assert(func_obj->func_kwdefaults == NULL);+ func_obj->func_kwdefaults = attr;+ break;+ case MAKE_FUNCTION_DEFAULTS:+ assert(PyTuple_CheckExact(attr));+ assert(func_obj->func_defaults == NULL);+ func_obj->func_defaults = attr;+ break;+ default:+ Py_UNREACHABLE();+ }+ STACK_SHRINK(1);+ stack_pointer[-1] = func;+ break;+ }++ case BUILD_SLICE: {+ PyObject *step = (oparg == 3) ? stack_pointer[-(((oparg == 3) ? 1 : 0))] : NULL;+ PyObject *stop = stack_pointer[-(1 + ((oparg == 3) ? 1 : 0))];+ PyObject *start = stack_pointer[-(2 + ((oparg == 3) ? 1 : 0))];+ PyObject *slice;+ slice = PySlice_New(start, stop, step);+ Py_DECREF(start);+ Py_DECREF(stop);+ Py_XDECREF(step);+ if (slice == NULL) { STACK_SHRINK(((oparg == 3) ? 1 : 0)); goto pop_2_error; }+ STACK_SHRINK(((oparg == 3) ? 1 : 0));+ STACK_SHRINK(1);+ stack_pointer[-1] = slice;+ break;+ }++ case CONVERT_VALUE: {+ PyObject *value = stack_pointer[-1];+ PyObject *result;+ convertion_func_ptr conv_fn;+ assert(oparg >= FVC_STR && oparg <= FVC_ASCII);+ conv_fn = CONVERSION_FUNCTIONS[oparg];+ result = conv_fn(value);+ Py_DECREF(value);+ if (result == NULL) goto pop_1_error;+ stack_pointer[-1] = result;+ break;+ }++ case FORMAT_SIMPLE: {+ PyObject *value = stack_pointer[-1];+ PyObject *res;+ /* If value is a unicode object, then we know the result+ * of format(value) is value itself. */+ if (!PyUnicode_CheckExact(value)) {+ res = PyObject_Format(value, NULL);+ Py_DECREF(value);+ if (res == NULL) goto pop_1_error;+ }+ else {+ res = value;+ }+ stack_pointer[-1] = res;+ break;+ }++ case FORMAT_WITH_SPEC: {+ PyObject *fmt_spec = stack_pointer[-1];+ PyObject *value = stack_pointer[-2];+ PyObject *res;+ res = PyObject_Format(value, fmt_spec);+ Py_DECREF(value);+ Py_DECREF(fmt_spec);+ if (res == NULL) goto pop_2_error;+ STACK_SHRINK(1);+ stack_pointer[-1] = res;+ break;+ }++ case COPY: {+ PyObject *bottom = stack_pointer[-(1 + (oparg-1))];+ PyObject *top;+ assert(oparg > 0);+ top = Py_NewRef(bottom);+ STACK_GROW(1);+ stack_pointer[-1] = top;+ break;+ }++ case SWAP: {+ PyObject *top = stack_pointer[-1];+ PyObject *bottom = stack_pointer[-(2 + (oparg-2))];+ assert(oparg >= 2);+ stack_pointer[-1] = bottom;+ stack_pointer[-(2 + (oparg-2))] = top;+ break;+ }diff --git a/Python/generated_cases.c.h b/Python/generated_cases.c.hindex 11ca535adfb19..43cfd4a882c73 100644--- a/Python/generated_cases.c.h+++ b/Python/generated_cases.c.h@@ -8,7 +8,7 @@ } TARGET(RESUME) {- #line 139 "Python/bytecodes.c"+ #line 137 "Python/bytecodes.c" assert(tstate->cframe == &cframe); assert(frame == cframe.current_frame); /* Possibly combine this with eval breaker */@@ -25,7 +25,7 @@ } TARGET(INSTRUMENTED_RESUME) {- #line 153 "Python/bytecodes.c"+ #line 151 "Python/bytecodes.c" /* Possible performance enhancement: * We need to check the eval breaker anyway, can we * combine the instrument verison check and the eval breaker test?@@ -57,7 +57,7 @@ TARGET(LOAD_CLOSURE) { PyObject *value;- #line 181 "Python/bytecodes.c"+ #line 179 "Python/bytecodes.c" /* We keep LOAD_CLOSURE so that the bytecode stays more readable. */ value = GETLOCAL(oparg); if (value == NULL) goto unbound_local_error;@@ -70,7 +70,7 @@ TARGET(LOAD_FAST_CHECK) { PyObject *value;- #line 188 "Python/bytecodes.c"+ #line 186 "Python/bytecodes.c" value = GETLOCAL(oparg); if (value == NULL) goto unbound_local_error; Py_INCREF(value);@@ -82,7 +82,7 @@ TARGET(LOAD_FAST) { PyObject *value;- #line 194 "Python/bytecodes.c"+ #line 192 "Python/bytecodes.c" value = GETLOCAL(oparg); assert(value != NULL); Py_INCREF(value);@@ -94,7 +94,7 @@ TARGET(LOAD_FAST_AND_CLEAR) { PyObject *value;- #line 200 "Python/bytecodes.c"+ #line 198 "Python/bytecodes.c" value = GETLOCAL(oparg); // do not use SETLOCAL here, it decrefs the old value GETLOCAL(oparg) = NULL;@@ -107,7 +107,7 @@ TARGET(LOAD_FAST_LOAD_FAST) { PyObject *value1; PyObject *value2;- #line 206 "Python/bytecodes.c"+ #line 204 "Python/bytecodes.c" uint32_t oparg1 = oparg >> 4; uint32_t oparg2 = oparg & 15; value1 = GETLOCAL(oparg1);@@ -123,7 +123,7 @@ TARGET(LOAD_CONST) { PyObject *value;- #line 215 "Python/bytecodes.c"+ #line 213 "Python/bytecodes.c" value = GETITEM(FRAME_CO_CONSTS, oparg); Py_INCREF(value); #line 130 "Python/generated_cases.c.h"@@ -134,7 +134,7 @@ TARGET(STORE_FAST) { PyObject *value = stack_pointer[-1];- #line 220 "Python/bytecodes.c"+ #line 218 "Python/bytecodes.c" SETLOCAL(oparg, value); #line 140 "Python/generated_cases.c.h" STACK_SHRINK(1);@@ -144,7 +144,7 @@ TARGET(STORE_FAST_LOAD_FAST) { PyObject *value1 = stack_pointer[-1]; PyObject *value2;- #line 228 "Python/bytecodes.c"+ #line 226 "Python/bytecodes.c" uint32_t oparg1 = oparg >> 4; uint32_t oparg2 = oparg & 15; SETLOCAL(oparg1, value1);@@ -158,7 +158,7 @@ TARGET(STORE_FAST_STORE_FAST) { PyObject *value1 = stack_pointer[-1]; PyObject *value2 = stack_pointer[-2];- #line 236 "Python/bytecodes.c"+ #line 234 "Python/bytecodes.c" uint32_t oparg1 = oparg >> 4; uint32_t oparg2 = oparg & 15; SETLOCAL(oparg1, value1);@@ -170,7 +170,7 @@ TARGET(POP_TOP) { PyObject *value = stack_pointer[-1];- #line 243 "Python/bytecodes.c"+ #line 241 "Python/bytecodes.c" #line 175 "Python/generated_cases.c.h" Py_DECREF(value); STACK_SHRINK(1);@@ -179,7 +179,7 @@ TARGET(PUSH_NULL) { PyObject *res;- #line 247 "Python/bytecodes.c"+ #line 245 "Python/bytecodes.c" res = NULL; #line 185 "Python/generated_cases.c.h" STACK_GROW(1);@@ -192,13 +192,13 @@ PyObject *_tmp_2 = stack_pointer[-2]; { PyObject *value = _tmp_1;- #line 243 "Python/bytecodes.c"+ #line 241 "Python/bytecodes.c" #line 197 "Python/generated_cases.c.h" Py_DECREF(value); } { PyObject *value = _tmp_2;- #line 243 "Python/bytecodes.c"+ #line 241 "Python/bytecodes.c" #line 203 "Python/generated_cases.c.h" Py_DECREF(value); }@@ -209,7 +209,7 @@ TARGET(INSTRUMENTED_END_FOR) { PyObject *value = stack_pointer[-1]; PyObject *receiver = stack_pointer[-2];- #line 253 "Python/bytecodes.c"+ #line 251 "Python/bytecodes.c" /* Need to create a fake StopIteration error here, * to conform to PEP 380 */ if (PyGen_Check(receiver)) {@@ -229,7 +229,7 @@ TARGET(END_SEND) { PyObject *value = stack_pointer[-1]; PyObject *receiver = stack_pointer[-2];- #line 266 "Python/bytecodes.c"+ #line 264 "Python/bytecodes.c" Py_DECREF(receiver); #line 235 "Python/generated_cases.c.h" STACK_SHRINK(1);@@ -240,7 +240,7 @@ TARGET(INSTRUMENTED_END_SEND) { PyObject *value = stack_pointer[-1]; PyObject *receiver = stack_pointer[-2];- #line 270 "Python/bytecodes.c"+ #line 268 "Python/bytecodes.c" if (PyGen_Check(receiver) || PyCoro_CheckExact(receiver)) { PyErr_SetObject(PyExc_StopIteration, value); if (monitor_stop_iteration(tstate, frame, next_instr-1)) {@@ -258,11 +258,11 @@ TARGET(UNARY_NEGATIVE) { PyObject *value = stack_pointer[-1]; PyObject *res;- #line 281 "Python/bytecodes.c"+ #line 279 "Python/bytecodes.c" res = PyNumber_Negative(value); #line 264 "Python/generated_cases.c.h" Py_DECREF(value);- #line 283 "Python/bytecodes.c"+ #line 281 "Python/bytecodes.c" if (res == NULL) goto pop_1_error; #line 268 "Python/generated_cases.c.h" stack_pointer[-1] = res;@@ -272,11 +272,11 @@ TARGET(UNARY_NOT) { PyObject *value = stack_pointer[-1]; PyObject *res;- #line 287 "Python/bytecodes.c"+ #line 285 "Python/bytecodes.c" int err = PyObject_IsTrue(value); #line 278 "Python/generated_cases.c.h" Py_DECREF(value);- #line 289 "Python/bytecodes.c"+ #line 287 "Python/bytecodes.c" if (err < 0) goto pop_1_error; if (err == 0) { res = Py_True;@@ -292,11 +292,11 @@ TARGET(UNARY_INVERT) { PyObject *value = stack_pointer[-1]; PyObject *res;- #line 299 "Python/bytecodes.c"+ #line 297 "Python/bytecodes.c" res = PyNumber_Invert(value); #line 298 "Python/generated_cases.c.h" Py_DECREF(value);- #line 301 "Python/bytecodes.c"+ #line 299 "Python/bytecodes.c" if (res == NULL) goto pop_1_error; #line 302 "Python/generated_cases.c.h" stack_pointer[-1] = res;@@ -309,7 +309,7 @@ { PyObject *right = _tmp_1; PyObject *left = _tmp_2;- #line 317 "Python/bytecodes.c"+ #line 315 "Python/bytecodes.c" DEOPT_IF(!PyLong_CheckExact(left), BINARY_OP); DEOPT_IF(!PyLong_CheckExact(right), BINARY_OP); #line 316 "Python/generated_cases.c.h"@@ -320,7 +320,7 @@ PyObject *right = _tmp_1; PyObject *left = _tmp_2; PyObject *res;- #line 322 "Python/bytecodes.c"+ #line 320 "Python/bytecodes.c" STAT_INC(BINARY_OP, hit); res = _PyLong_Multiply((PyLongObject *)left, (PyLongObject *)right); _Py_DECREF_SPECIALIZED(right, (destructor)PyObject_Free);@@ -341,7 +341,7 @@ { PyObject *right = _tmp_1; PyObject *left = _tmp_2;- #line 317 "Python/bytecodes.c"+ #line 315 "Python/bytecodes.c" DEOPT_IF(!PyLong_CheckExact(left), BINARY_OP); DEOPT_IF(!PyLong_CheckExact(right), BINARY_OP); #line 348 "Python/generated_cases.c.h"@@ -352,7 +352,7 @@ PyObject *right = _tmp_1; PyObject *left = _tmp_2; PyObject *res;- #line 330 "Python/bytecodes.c"+ #line 328 "Python/bytecodes.c" STAT_INC(BINARY_OP, hit); res = _PyLong_Add((PyLongObject *)left, (PyLongObject *)right); _Py_DECREF_SPECIALIZED(right, (destructor)PyObject_Free);@@ -373,7 +373,7 @@ { PyObject *right = _tmp_1; PyObject *left = _tmp_2;- #line 317 "Python/bytecodes.c"+ #line 315 "Python/bytecodes.c" DEOPT_IF(!PyLong_CheckExact(left), BINARY_OP); DEOPT_IF(!PyLong_CheckExact(right), BINARY_OP); #line 380 "Python/generated_cases.c.h"@@ -384,7 +384,7 @@ PyObject *right = _tmp_1; PyObject *left = _tmp_2; PyObject *res;- #line 338 "Python/bytecodes.c"+ #line 336 "Python/bytecodes.c" STAT_INC(BINARY_OP, hit); res = _PyLong_Subtract((PyLongObject *)left, (PyLongObject *)right); _Py_DECREF_SPECIALIZED(right, (destructor)PyObject_Free);@@ -405,7 +405,7 @@ { PyObject *right = _tmp_1; PyObject *left = _tmp_2;- #line 353 "Python/bytecodes.c"+ #line 351 "Python/bytecodes.c" DEOPT_IF(!PyFloat_CheckExact(left), BINARY_OP); DEOPT_IF(!PyFloat_CheckExact(right), BINARY_OP); #line 412 "Python/generated_cases.c.h"@@ -416,7 +416,7 @@ PyObject *right = _tmp_1; PyObject *left = _tmp_2; PyObject *res;- #line 358 "Python/bytecodes.c"+ #line 356 "Python/bytecodes.c" STAT_INC(BINARY_OP, hit); double dres = ((PyFloatObject *)left)->ob_fval *@@ -437,7 +437,7 @@ { PyObject *right = _tmp_1; PyObject *left = _tmp_2;- #line 353 "Python/bytecodes.c"+ #line 351 "Python/bytecodes.c" DEOPT_IF(!PyFloat_CheckExact(left), BINARY_OP); DEOPT_IF(!PyFloat_CheckExact(right), BINARY_OP); #line 444 "Python/generated_cases.c.h"@@ -448,7 +448,7 @@ PyObject *right = _tmp_1; PyObject *left = _tmp_2; PyObject *res;- #line 366 "Python/bytecodes.c"+ #line 364 "Python/bytecodes.c" STAT_INC(BINARY_OP, hit); double dres = ((PyFloatObject *)left)->ob_fval +@@ -469,7 +469,7 @@ { PyObject *right = _tmp_1; PyObject *left = _tmp_2;- #line 353 "Python/bytecodes.c"+ #line 351 "Python/bytecodes.c" DEOPT_IF(!PyFloat_CheckExact(left), BINARY_OP); DEOPT_IF(!PyFloat_CheckExact(right), BINARY_OP); #line 476 "Python/generated_cases.c.h"@@ -480,7 +480,7 @@ PyObject *right = _tmp_1; PyObject *left = _tmp_2; PyObject *res;- #line 374 "Python/bytecodes.c"+ #line 372 "Python/bytecodes.c" STAT_INC(BINARY_OP, hit); double dres = ((PyFloatObject *)left)->ob_fval -@@ -501,7 +501,7 @@ { PyObject *right = _tmp_1; PyObject *left = _tmp_2;- #line 389 "Python/bytecodes.c"+ #line 387 "Python/bytecodes.c" DEOPT_IF(!PyUnicode_CheckExact(left), BINARY_OP); DEOPT_IF(!PyUnicode_CheckExact(right), BINARY_OP); #line 508 "Python/generated_cases.c.h"@@ -512,7 +512,7 @@ PyObject *right = _tmp_1; PyObject *left = _tmp_2; PyObject *res;- #line 394 "Python/bytecodes.c"+ #line 392 "Python/bytecodes.c" STAT_INC(BINARY_OP, hit); res = PyUnicode_Concat(left, right); _Py_DECREF_SPECIALIZED(left, _PyUnicode_ExactDealloc);@@ -533,7 +533,7 @@ { PyObject *right = _tmp_1; PyObject *left = _tmp_2;- #line 389 "Python/bytecodes.c"+ #line 387 "Python/bytecodes.c" DEOPT_IF(!PyUnicode_CheckExact(left), BINARY_OP); DEOPT_IF(!PyUnicode_CheckExact(right), BINARY_OP); #line 540 "Python/generated_cases.c.h"@@ -543,7 +543,7 @@ { PyObject *right = _tmp_1; PyObject *left = _tmp_2;- #line 411 "Python/bytecodes.c"+ #line 409 "Python/bytecodes.c" _Py_CODEUNIT true_next = next_instr[INLINE_CACHE_ENTRIES_BINARY_OP]; assert(true_next.op.code == STORE_FAST); PyObject **target_local = &GETLOCAL(true_next.op.arg);@@ -579,7 +579,7 @@ PyObject *sub = stack_pointer[-1]; PyObject *container = stack_pointer[-2]; PyObject *res;- #line 448 "Python/bytecodes.c"+ #line 446 "Python/bytecodes.c" #if ENABLE_SPECIALIZATION _PyBinarySubscrCache *cache = (_PyBinarySubscrCache *)next_instr; if (ADAPTIVE_COUNTER_IS_ZERO(cache->counter)) {@@ -594,7 +594,7 @@ #line 595 "Python/generated_cases.c.h" Py_DECREF(container); Py_DECREF(sub);- #line 460 "Python/bytecodes.c"+ #line 458 "Python/bytecodes.c" if (res == NULL) goto pop_2_error; #line 600 "Python/generated_cases.c.h" STACK_SHRINK(1);@@ -608,7 +608,7 @@ PyObject *start = stack_pointer[-2]; PyObject *container = stack_pointer[-3]; PyObject *res;- #line 464 "Python/bytecodes.c"+ #line 462 "Python/bytecodes.c" PyObject *slice = _PyBuildSlice_ConsumeRefs(start, stop); // Can't use ERROR_IF() here, because we haven't // DECREF'ed container yet, and we still own slice.@@ -632,7 +632,7 @@ PyObject *start = stack_pointer[-2]; PyObject *container = stack_pointer[-3]; PyObject *v = stack_pointer[-4];- #line 479 "Python/bytecodes.c"+ #line 477 "Python/bytecodes.c" PyObject *slice = _PyBuildSlice_ConsumeRefs(start, stop); int err; if (slice == NULL) {@@ -654,7 +654,7 @@ PyObject *sub = stack_pointer[-1]; PyObject *list = stack_pointer[-2]; PyObject *res;- #line 494 "Python/bytecodes.c"+ #line 492 "Python/bytecodes.c" DEOPT_IF(!PyLong_CheckExact(sub), BINARY_SUBSCR); DEOPT_IF(!PyList_CheckExact(list), BINARY_SUBSCR); @@ -679,7 +679,7 @@ PyObject *sub = stack_pointer[-1]; PyObject *tuple = stack_pointer[-2]; PyObject *res;- #line 510 "Python/bytecodes.c"+ #line 508 "Python/bytecodes.c" DEOPT_IF(!PyLong_CheckExact(sub), BINARY_SUBSCR); DEOPT_IF(!PyTuple_CheckExact(tuple), BINARY_SUBSCR); @@ -704,7 +704,7 @@ PyObject *sub = stack_pointer[-1]; PyObject *dict = stack_pointer[-2]; PyObject *res;- #line 526 "Python/bytecodes.c"+ #line 524 "Python/bytecodes.c" DEOPT_IF(!PyDict_CheckExact(dict), BINARY_SUBSCR); STAT_INC(BINARY_SUBSCR, hit); res = PyDict_GetItemWithError(dict, sub);@@ -715,7 +715,7 @@ #line 716 "Python/generated_cases.c.h" Py_DECREF(dict); Py_DECREF(sub);- #line 534 "Python/bytecodes.c"+ #line 532 "Python/bytecodes.c" if (true) goto pop_2_error; } Py_INCREF(res); // Do this before DECREF'ing dict, sub@@ -731,7 +731,7 @@ TARGET(BINARY_SUBSCR_GETITEM) { PyObject *sub = stack_pointer[-1]; PyObject *container = stack_pointer[-2];- #line 541 "Python/bytecodes.c"+ #line 539 "Python/bytecodes.c" DEOPT_IF(tstate->interp->eval_frame, BINARY_SUBSCR); PyTypeObject *tp = Py_TYPE(container); DEOPT_IF(!PyType_HasFeature(tp, Py_TPFLAGS_HEAPTYPE), BINARY_SUBSCR);@@ -760,7 +760,7 @@ TARGET(LIST_APPEND) { PyObject *v = stack_pointer[-1]; PyObject *list = stack_pointer[-(2 + (oparg-1))];- #line 566 "Python/bytecodes.c"+ #line 564 "Python/bytecodes.c" if (_PyList_AppendTakeRef((PyListObject *)list, v) < 0) goto pop_1_error; #line 766 "Python/generated_cases.c.h" STACK_SHRINK(1);@@ -770,11 +770,11 @@ TARGET(SET_ADD) { PyObject *v = stack_pointer[-1]; PyObject *set = stack_pointer[-(2 + (oparg-1))];- #line 570 "Python/bytecodes.c"+ #line 568 "Python/bytecodes.c" int err = PySet_Add(set, v); #line 776 "Python/generated_cases.c.h" Py_DECREF(v);- #line 572 "Python/bytecodes.c"+ #line 570 "Python/bytecodes.c" if (err) goto pop_1_error; #line 780 "Python/generated_cases.c.h" STACK_SHRINK(1);@@ -788,7 +788,7 @@ PyObject *container = stack_pointer[-2]; PyObject *v = stack_pointer[-3]; uint16_t counter = read_u16(&next_instr[0].cache);- #line 582 "Python/bytecodes.c"+ #line 580 "Python/bytecodes.c" #if ENABLE_SPECIALIZATION if (ADAPTIVE_COUNTER_IS_ZERO(counter)) { next_instr--;@@ -807,7 +807,7 @@ Py_DECREF(v); Py_DECREF(container); Py_DECREF(sub);- #line 597 "Python/bytecodes.c"+ #line 595 "Python/bytecodes.c" if (err) goto pop_3_error; #line 813 "Python/generated_cases.c.h" STACK_SHRINK(3);@@ -819,7 +819,7 @@ PyObject *sub = stack_pointer[-1]; PyObject *list = stack_pointer[-2]; PyObject *value = stack_pointer[-3];- #line 601 "Python/bytecodes.c"+ #line 599 "Python/bytecodes.c" DEOPT_IF(!PyLong_CheckExact(sub), STORE_SUBSCR); DEOPT_IF(!PyList_CheckExact(list), STORE_SUBSCR); @@ -846,7 +846,7 @@ PyObject *sub = stack_pointer[-1]; PyObject *dict = stack_pointer[-2]; PyObject *value = stack_pointer[-3];- #line 620 "Python/bytecodes.c"+ #line 618 "Python/bytecodes.c" DEOPT_IF(!PyDict_CheckExact(dict), STORE_SUBSCR); STAT_INC(STORE_SUBSCR, hit); int err = _PyDict_SetItem_Take2((PyDictObject *)dict, sub, value);@@ -861,13 +861,13 @@ TARGET(DELETE_SUBSCR) { PyObject *sub = stack_pointer[-1]; PyObject *container = stack_pointer[-2];- #line 628 "Python/bytecodes.c"+ #line 626 "Python/bytecodes.c" /* del container[sub] */ int err = PyObject_DelItem(container, sub); #line 868 "Python/generated_cases.c.h" Py_DECREF(container); Py_DECREF(sub);- #line 631 "Python/bytecodes.c"+ #line 629 "Python/bytecodes.c" if (err) goto pop_2_error; #line 873 "Python/generated_cases.c.h" STACK_SHRINK(2);@@ -877,12 +877,12 @@ TARGET(CALL_INTRINSIC_1) { PyObject *value = stack_pointer[-1]; PyObject *res;- #line 635 "Python/bytecodes.c"+ #line 633 "Python/bytecodes.c" assert(oparg <= MAX_INTRINSIC_1); res = _PyIntrinsics_UnaryFunctions[oparg](tstate, value); #line 884 "Python/generated_cases.c.h" Py_DECREF(value);- #line 638 "Python/bytecodes.c"+ #line 636 "Python/bytecodes.c" if (res == NULL) goto pop_1_error; #line 888 "Python/generated_cases.c.h" stack_pointer[-1] = res;@@ -893,13 +893,13 @@ PyObject *value1 = stack_pointer[-1]; PyObject *value2 = stack_pointer[-2]; PyObject *res;- #line 642 "Python/bytecodes.c"+ #line 640 "Python/bytecodes.c" assert(oparg <= MAX_INTRINSIC_2); res = _PyIntrinsics_BinaryFunctions[oparg](tstate, value2, value1); #line 900 "Python/generated_cases.c.h" Py_DECREF(value2); Py_DECREF(value1);- #line 645 "Python/bytecodes.c"+ #line 643 "Python/bytecodes.c" if (res == NULL) goto pop_2_error; #line 905 "Python/generated_cases.c.h" STACK_SHRINK(1);@@ -909,7 +909,7 @@ TARGET(RAISE_VARARGS) { PyObject **args = (stack_pointer - oparg);- #line 649 "Python/bytecodes.c"+ #line 647 "Python/bytecodes.c" PyObject *cause = NULL, *exc = NULL; switch (oparg) { case 2:@@ -932,7 +932,7 @@ TARGET(INTERPRETER_EXIT) { PyObject *retval = stack_pointer[-1];- #line 669 "Python/bytecodes.c"+ #line 667 "Python/bytecodes.c" assert(frame == &entry_frame); assert(_PyFrame_IsIncomplete(frame)); /* Restore previous cframe and return. */@@ -946,7 +946,7 @@ TARGET(RETURN_VALUE) { PyObject *retval = stack_pointer[-1];- #line 680 "Python/bytecodes.c"+ #line 678 "Python/bytecodes.c" STACK_SHRINK(1); assert(EMPTY()); _PyFrame_SetStackPointer(frame, stack_pointer);@@ -964,7 +964,7 @@ TARGET(INSTRUMENTED_RETURN_VALUE) { PyObject *retval = stack_pointer[-1];- #line 695 "Python/bytecodes.c"+ #line 693 "Python/bytecodes.c" int err = _Py_call_instrumentation_arg( tstate, PY_MONITORING_EVENT_PY_RETURN, frame, next_instr-1, retval);@@ -985,7 +985,7 @@ } TARGET(RETURN_CONST) {- #line 714 "Python/bytecodes.c"+ #line 712 "Python/bytecodes.c" PyObject *retval = GETITEM(FRAME_CO_CONSTS, oparg); Py_INCREF(retval); assert(EMPTY());@@ -1003,7 +1003,7 @@ } TARGET(INSTRUMENTED_RETURN_CONST) {- #line 730 "Python/bytecodes.c"+ #line 728 "Python/bytecodes.c" PyObject *retval = GETITEM(FRAME_CO_CONSTS, oparg); int err = _Py_call_instrumentation_arg( tstate, PY_MONITORING_EVENT_PY_RETURN,@@ -1027,7 +1027,7 @@ TARGET(GET_AITER) { PyObject *obj = stack_pointer[-1]; PyObject *iter;- #line 750 "Python/bytecodes.c"+ #line 748 "Python/bytecodes.c" unaryfunc getter = NULL; PyTypeObject *type = Py_TYPE(obj); @@ -1042,14 +1042,14 @@ type->tp_name); #line 1044 "Python/generated_cases.c.h" Py_DECREF(obj);- #line 763 "Python/bytecodes.c"+ #line 761 "Python/bytecodes.c" if (true) goto pop_1_error; } iter = (*getter)(obj); #line 1051 "Python/generated_cases.c.h" Py_DECREF(obj);- #line 768 "Python/bytecodes.c"+ #line 766 "Python/bytecodes.c" if (iter == NULL) goto pop_1_error; if (Py_TYPE(iter)->tp_as_async == NULL ||@@ -1070,7 +1070,7 @@ TARGET(GET_ANEXT) { PyObject *aiter = stack_pointer[-1]; PyObject *awaitable;- #line 783 "Python/bytecodes.c"+ #line 781 "Python/bytecodes.c" unaryfunc getter = NULL; PyObject *next_iter = NULL; PyTypeObject *type = Py_TYPE(aiter);@@ -1122,7 +1122,7 @@ TARGET(GET_AWAITABLE) { PyObject *iterable = stack_pointer[-1]; PyObject *iter;- #line 828 "Python/bytecodes.c"+ #line 826 "Python/bytecodes.c" iter = _PyCoro_GetAwaitableIter(iterable); if (iter == NULL) {@@ -1131,7 +1131,7 @@ #line 1133 "Python/generated_cases.c.h" Py_DECREF(iterable);- #line 835 "Python/bytecodes.c"+ #line 833 "Python/bytecodes.c" if (iter != NULL && PyCoro_CheckExact(iter)) { PyObject *yf = _PyGen_yf((PyGenObject*)iter);@@ -1159,7 +1159,7 @@ PyObject *v = stack_pointer[-1]; PyObject *receiver = stack_pointer[-2]; PyObject *retval;- #line 859 "Python/bytecodes.c"+ #line 857 "Python/bytecodes.c" #if ENABLE_SPECIALIZATION _PySendCache *cache = (_PySendCache *)next_instr; if (ADAPTIVE_COUNTER_IS_ZERO(cache->counter)) {@@ -1215,7 +1215,7 @@ TARGET(SEND_GEN) { PyObject *v = stack_pointer[-1]; PyObject *receiver = stack_pointer[-2];- #line 908 "Python/bytecodes.c"+ #line 906 "Python/bytecodes.c" DEOPT_IF(tstate->interp->eval_frame, SEND); PyGenObject *gen = (PyGenObject *)receiver; DEOPT_IF(Py_TYPE(gen) != &PyGen_Type &&@@ -1236,7 +1236,7 @@ TARGET(INSTRUMENTED_YIELD_VALUE) { PyObject *retval = stack_pointer[-1];- #line 926 "Python/bytecodes.c"+ #line 924 "Python/bytecodes.c" assert(frame != &entry_frame); assert(oparg >= 0); /* make the generator identify this as HAS_ARG */ PyGenObject *gen = _PyFrame_GetGenerator(frame);@@ -1259,7 +1259,7 @@ TARGET(YIELD_VALUE) { PyObject *retval = stack_pointer[-1];- #line 946 "Python/bytecodes.c"+ #line 944 "Python/bytecodes.c" // NOTE: It's important that YIELD_VALUE never raises an exception! // The compiler treats any exception raised here as a failed close() // or throw() call.@@ -1281,7 +1281,7 @@ TARGET(POP_EXCEPT) { PyObject *exc_value = stack_pointer[-1];- #line 965 "Python/bytecodes.c"+ #line 963 "Python/bytecodes.c" _PyErr_StackItem *exc_info = tstate->exc_info; Py_XSETREF(exc_info->exc_value, exc_value); #line 1288 "Python/generated_cases.c.h"@@ -1292,7 +1292,7 @@ TARGET(RERAISE) { PyObject *exc = stack_pointer[-1]; PyObject **values = (stack_pointer - (1 + oparg));- #line 970 "Python/bytecodes.c"+ #line 968 "Python/bytecodes.c" assert(oparg >= 0 && oparg <= 2); if (oparg) { PyObject *lasti = values[0];@@ -1316,13 +1316,13 @@ TARGET(END_ASYNC_FOR) { PyObject *exc = stack_pointer[-1]; PyObject *awaitable = stack_pointer[-2];- #line 990 "Python/bytecodes.c"+ #line 988 "Python/bytecodes.c" assert(exc && PyExceptionInstance_Check(exc)); if (PyErr_GivenExceptionMatches(exc, PyExc_StopAsyncIteration)) { #line 1323 "Python/generated_cases.c.h" Py_DECREF(awaitable); Py_DECREF(exc);- #line 993 "Python/bytecodes.c"+ #line 991 "Python/bytecodes.c" } else { Py_INCREF(exc);@@ -1340,7 +1340,7 @@ PyObject *sub_iter = stack_pointer[-3]; PyObject *none; PyObject *value;- #line 1002 "Python/bytecodes.c"+ #line 1000 "Python/bytecodes.c" assert(throwflag); assert(exc_value && PyExceptionInstance_Check(exc_value)); if (PyErr_GivenExceptionMatches(exc_value, PyExc_StopIteration)) {@@ -1349,7 +1349,7 @@ Py_DECREF(sub_iter); Py_DECREF(last_sent_val); Py_DECREF(exc_value);- #line 1007 "Python/bytecodes.c"+ #line 1005 "Python/bytecodes.c" none = Py_None; } else {@@ -1365,7 +1365,7 @@ TARGET(LOAD_ASSERTION_ERROR) { PyObject *value;- #line 1016 "Python/bytecodes.c"+ #line 1014 "Python/bytecodes.c" value = Py_NewRef(PyExc_AssertionError); #line 1371 "Python/generated_cases.c.h" STACK_GROW(1);@@ -1375,7 +1375,7 @@ TARGET(LOAD_BUILD_CLASS) { PyObject *bc;- #line 1020 "Python/bytecodes.c"+ #line 1018 "Python/bytecodes.c" if (PyDict_CheckExact(BUILTINS())) { bc = _PyDict_GetItemWithError(BUILTINS(), &_Py_ID(__build_class__));@@ -1405,7 +1405,7 @@ TARGET(STORE_NAME) { PyObject *v = stack_pointer[-1];- #line 1045 "Python/bytecodes.c"+ #line 1043 "Python/bytecodes.c" PyObject *name = GETITEM(FRAME_CO_NAMES, oparg); PyObject *ns = LOCALS(); int err;@@ -1414,7 +1414,7 @@ "no locals found when storing %R", name); #line 1416 "Python/generated_cases.c.h" Py_DECREF(v);- #line 1052 "Python/bytecodes.c"+ #line 1050 "Python/bytecodes.c" if (true) goto pop_1_error; } if (PyDict_CheckExact(ns))@@ -1423,7 +1423,7 @@ err = PyObject_SetItem(ns, name, v); #line 1425 "Python/generated_cases.c.h" Py_DECREF(v);- #line 1059 "Python/bytecodes.c"+ #line 1057 "Python/bytecodes.c" if (err) goto pop_1_error; #line 1429 "Python/generated_cases.c.h" STACK_SHRINK(1);@@ -1431,7 +1431,7 @@ } TARGET(DELETE_NAME) {- #line 1063 "Python/bytecodes.c"+ #line 1061 "Python/bytecodes.c" PyObject *name = GETITEM(FRAME_CO_NAMES, oparg); PyObject *ns = LOCALS(); int err;@@ -1456,7 +1456,7 @@ PREDICTED(UNPACK_SEQUENCE); static_assert(INLINE_CACHE_ENTRIES_UNPACK_SEQUENCE == 1, "incorrect cache size"); PyObject *seq = stack_pointer[-1];- #line 1089 "Python/bytecodes.c"+ #line 1087 "Python/bytecodes.c" #if ENABLE_SPECIALIZATION _PyUnpackSequenceCache *cache = (_PyUnpackSequenceCache *)next_instr; if (ADAPTIVE_COUNTER_IS_ZERO(cache->counter)) {@@ -1471,7 +1471,7 @@ int res = unpack_iterable(tstate, seq, oparg, -1, top); #line 1473 "Python/generated_cases.c.h" Py_DECREF(seq);- #line 1102 "Python/bytecodes.c"+ #line 1100 "Python/bytecodes.c" if (res == 0) goto pop_1_error; #line 1477 "Python/generated_cases.c.h" STACK_SHRINK(1);@@ -1483,7 +1483,7 @@ TARGET(UNPACK_SEQUENCE_TWO_TUPLE) { PyObject *seq = stack_pointer[-1]; PyObject **values = stack_pointer - (1);- #line 1106 "Python/bytecodes.c"+ #line 1104 "Python/bytecodes.c" DEOPT_IF(!PyTuple_CheckExact(seq), UNPACK_SEQUENCE); DEOPT_IF(PyTuple_GET_SIZE(seq) != 2, UNPACK_SEQUENCE); assert(oparg == 2);@@ -1501,7 +1501,7 @@ TARGET(UNPACK_SEQUENCE_TUPLE) { PyObject *seq = stack_pointer[-1]; PyObject **values = stack_pointer - (1);- #line 1116 "Python/bytecodes.c"+ #line 1114 "Python/bytecodes.c" DEOPT_IF(!PyTuple_CheckExact(seq), UNPACK_SEQUENCE); DEOPT_IF(PyTuple_GET_SIZE(seq) != oparg, UNPACK_SEQUENCE); STAT_INC(UNPACK_SEQUENCE, hit);@@ -1520,7 +1520,7 @@ TARGET(UNPACK_SEQUENCE_LIST) { PyObject *seq = stack_pointer[-1]; PyObject **values = stack_pointer - (1);- #line 1127 "Python/bytecodes.c"+ #line 1125 "Python/bytecodes.c" DEOPT_IF(!PyList_CheckExact(seq), UNPACK_SEQUENCE); DEOPT_IF(PyList_GET_SIZE(seq) != oparg, UNPACK_SEQUENCE); STAT_INC(UNPACK_SEQUENCE, hit);@@ -1538,13 +1538,13 @@ TARGET(UNPACK_EX) { PyObject *seq = stack_pointer[-1];- #line 1138 "Python/bytecodes.c"+ #line 1136 "Python/bytecodes.c" int totalargs = 1 + (oparg & 0xFF) + (oparg >> 8); PyObject **top = stack_pointer + totalargs - 1; int res = unpack_iterable(tstate, seq, oparg & 0xFF, oparg >> 8, top); #line 1546 "Python/generated_cases.c.h" Py_DECREF(seq);- #line 1142 "Python/bytecodes.c"+ #line 1140 "Python/bytecodes.c" if (res == 0) goto pop_1_error; #line 1550 "Python/generated_cases.c.h" STACK_GROW((oparg & 0xFF) + (oparg >> 8));@@ -1557,7 +1557,7 @@ PyObject *owner = stack_pointer[-1]; PyObject *v = stack_pointer[-2]; uint16_t counter = read_u16(&next_instr[0].cache);- #line 1153 "Python/bytecodes.c"+ #line 1151 "Python/bytecodes.c" #if ENABLE_SPECIALIZATION if (ADAPTIVE_COUNTER_IS_ZERO(counter)) { PyObject *name = GETITEM(FRAME_CO_NAMES, oparg);@@ -1576,7 +1576,7 @@ #line 1577 "Python/generated_cases.c.h" Py_DECREF(v); Py_DECREF(owner);- #line 1169 "Python/bytecodes.c"+ #line 1167 "Python/bytecodes.c" if (err) goto pop_2_error; #line 1582 "Python/generated_cases.c.h" STACK_SHRINK(2);@@ -1586,12 +1586,12 @@ TARGET(DELETE_ATTR) { PyObject *owner = stack_pointer[-1];- #line 1173 "Python/bytecodes.c"+ #line 1171 "Python/bytecodes.c" PyObject *name = GETITEM(FRAME_CO_NAMES, oparg); int err = PyObject_SetAttr(owner, name, (PyObject *)NULL); #line 1593 "Python/generated_cases.c.h" Py_DECREF(owner);- #line 1176 "Python/bytecodes.c"+ #line 1174 "Python/bytecodes.c" if (err) goto pop_1_error; #line 1597 "Python/generated_cases.c.h" STACK_SHRINK(1);@@ -1600,12 +1600,12 @@ TARGET(STORE_GLOBAL) { PyObject *v = stack_pointer[-1];- #line 1180 "Python/bytecodes.c"+ #line 1178 "Python/bytecodes.c" PyObject *name = GETITEM(FRAME_CO_NAMES, oparg); int err = PyDict_SetItem(GLOBALS(), name, v); #line 1607 "Python/generated_cases.c.h" Py_DECREF(v);- #line 1183 "Python/bytecodes.c"+ #line 1181 "Python/bytecodes.c" if (err) goto pop_1_error; #line 1611 "Python/generated_cases.c.h" STACK_SHRINK(1);@@ -1613,7 +1613,7 @@ } TARGET(DELETE_GLOBAL) {- #line 1187 "Python/bytecodes.c"+ #line 1185 "Python/bytecodes.c" PyObject *name = GETITEM(FRAME_CO_NAMES, oparg); int err; err = PyDict_DelItem(GLOBALS(), name);@@ -1633,7 +1633,7 @@ PyObject *_tmp_1; { PyObject *locals;- #line 1201 "Python/bytecodes.c"+ #line 1199 "Python/bytecodes.c" locals = LOCALS(); if (locals == NULL) { _PyErr_SetString(tstate, PyExc_SystemError,@@ -1653,7 +1653,7 @@ PyObject *_tmp_1; { PyObject *locals;- #line 1201 "Python/bytecodes.c"+ #line 1199 "Python/bytecodes.c" locals = LOCALS(); if (locals == NULL) { _PyErr_SetString(tstate, PyExc_SystemError,@@ -1667,7 +1667,7 @@ { PyObject *mod_or_class_dict = _tmp_1; PyObject *v;- #line 1213 "Python/bytecodes.c"+ #line 1211 "Python/bytecodes.c" PyObject *name = GETITEM(FRAME_CO_NAMES, oparg); if (PyDict_CheckExact(mod_or_class_dict)) { v = PyDict_GetItemWithError(mod_or_class_dict, name);@@ -1737,7 +1737,7 @@ { PyObject *mod_or_class_dict = _tmp_1; PyObject *v;- #line 1213 "Python/bytecodes.c"+ #line 1211 "Python/bytecodes.c" PyObject *name = GETITEM(FRAME_CO_NAMES, oparg); if (PyDict_CheckExact(mod_or_class_dict)) { v = PyDict_GetItemWithError(mod_or_class_dict, name);@@ -1806,7 +1806,7 @@ static_assert(INLINE_CACHE_ENTRIES_LOAD_GLOBAL == 4, "incorrect cache size"); PyObject *null = NULL; PyObject *v;- #line 1282 "Python/bytecodes.c"+ #line 1280 "Python/bytecodes.c" #if ENABLE_SPECIALIZATION _PyLoadGlobalCache *cache = (_PyLoadGlobalCache *)next_instr; if (ADAPTIVE_COUNTER_IS_ZERO(cache->counter)) {@@ -1872,7 +1872,7 @@ PyObject *res; uint16_t index = read_u16(&next_instr[1].cache); uint16_t version = read_u16(&next_instr[2].cache);- #line 1336 "Python/bytecodes.c"+ #line 1334 "Python/bytecodes.c" DEOPT_IF(!PyDict_CheckExact(GLOBALS()), LOAD_GLOBAL); PyDictObject *dict = (PyDictObject *)GLOBALS(); DEOPT_IF(dict->ma_keys->dk_version != version, LOAD_GLOBAL);@@ -1898,7 +1898,7 @@ uint16_t index = read_u16(&next_instr[1].cache); uint16_t mod_version = read_u16(&next_instr[2].cache); uint16_t bltn_version = read_u16(&next_instr[3].cache);- #line 1349 "Python/bytecodes.c"+ #line 1347 "Python/bytecodes.c" DEOPT_IF(!PyDict_CheckExact(GLOBALS()), LOAD_GLOBAL); DEOPT_IF(!PyDict_CheckExact(BUILTINS()), LOAD_GLOBAL); PyDictObject *mdict = (PyDictObject *)GLOBALS();@@ -1923,7 +1923,7 @@ } TARGET(DELETE_FAST) {- #line 1366 "Python/bytecodes.c"+ #line 1364 "Python/bytecodes.c" PyObject *v = GETLOCAL(oparg); if (v == NULL) goto unbound_local_error; SETLOCAL(oparg, NULL);@@ -1932,7 +1932,7 @@ } TARGET(MAKE_CELL) {- #line 1372 "Python/bytecodes.c"+ #line 1370 "Python/bytecodes.c" // "initial" is probably NULL but not if it's an arg (or set // via PyFrame_LocalsToFast() before MAKE_CELL has run). PyObject *initial = GETLOCAL(oparg);@@ -1946,7 +1946,7 @@ } TARGET(DELETE_DEREF) {- #line 1383 "Python/bytecodes.c"+ #line 1381 "Python/bytecodes.c" PyObject *cell = GETLOCAL(oparg); PyObject *oldobj = PyCell_GET(cell); // Can't use ERROR_IF here.@@ -1964,7 +1964,7 @@ TARGET(LOAD_FROM_DICT_OR_DEREF) { PyObject *class_dict = stack_pointer[-1]; PyObject *value;- #line 1396 "Python/bytecodes.c"+ #line 1394 "Python/bytecodes.c" PyObject *name; assert(class_dict); assert(oparg >= 0 && oparg < _PyFrame_GetCode(frame)->co_nlocalsplus);@@ -2006,7 +2006,7 @@ TARGET(LOAD_DEREF) { PyObject *value;- #line 1433 "Python/bytecodes.c"+ #line 1431 "Python/bytecodes.c" PyObject *cell = GETLOCAL(oparg); value = PyCell_GET(cell); if (value == NULL) {@@ -2022,7 +2022,7 @@ TARGET(STORE_DEREF) { PyObject *v = stack_pointer[-1];- #line 1443 "Python/bytecodes.c"+ #line 1441 "Python/bytecodes.c" PyObject *cell = GETLOCAL(oparg); PyObject *oldobj = PyCell_GET(cell); PyCell_SET(cell, v);@@ -2033,7 +2033,7 @@ } TARGET(COPY_FREE_VARS) {- #line 1450 "Python/bytecodes.c"+ #line 1448 "Python/bytecodes.c" /* Copy closure variables to free variables */ PyCodeObject *co = _PyFrame_GetCode(frame); assert(PyFunction_Check(frame->f_funcobj));@@ -2051,13 +2051,13 @@ TARGET(BUILD_STRING) { PyObject **pieces = (stack_pointer - oparg); PyObject *str;- #line 1463 "Python/bytecodes.c"+ #line 1461 "Python/bytecodes.c" str = _PyUnicode_JoinArray(&_Py_STR(empty), pieces, oparg); #line 2057 "Python/generated_cases.c.h" for (int _i = oparg; --_i >= 0;) { Py_DECREF(pieces[_i]); }- #line 1465 "Python/bytecodes.c"+ #line 1463 "Python/bytecodes.c" if (str == NULL) { STACK_SHRINK(oparg); goto error; } #line 2063 "Python/generated_cases.c.h" STACK_SHRINK(oparg);@@ -2069,7 +2069,7 @@ TARGET(BUILD_TUPLE) { PyObject **values = (stack_pointer - oparg); PyObject *tup;- #line 1469 "Python/bytecodes.c"+ #line 1467 "Python/bytecodes.c" tup = _PyTuple_FromArraySteal(values, oparg); if (tup == NULL) { STACK_SHRINK(oparg); goto error; } #line 2076 "Python/generated_cases.c.h"@@ -2082,7 +2082,7 @@ TARGET(BUILD_LIST) { PyObject **values = (stack_pointer - oparg); PyObject *list;- #line 1474 "Python/bytecodes.c"+ #line 1472 "Python/bytecodes.c" list = _PyList_FromArraySteal(values, oparg); if (list == NULL) { STACK_SHRINK(oparg); goto error; } #line 2089 "Python/generated_cases.c.h"@@ -2095,7 +2095,7 @@ TARGET(LIST_EXTEND) { PyObject *iterable = stack_pointer[-1]; PyObject *list = stack_pointer[-(2 + (oparg-1))];- #line 1479 "Python/bytecodes.c"+ #line 1477 "Python/bytecodes.c" PyObject *none_val = _PyList_Extend((PyListObject *)list, iterable); if (none_val == NULL) { if (_PyErr_ExceptionMatches(tstate, PyExc_TypeError) &&@@ -2108,7 +2108,7 @@ } #line 2110 "Python/generated_cases.c.h" Py_DECREF(iterable);- #line 1490 "Python/bytecodes.c"+ #line 1488 "Python/bytecodes.c" if (true) goto pop_1_error; } assert(Py_IsNone(none_val));@@ -2121,11 +2121,11 @@ TARGET(SET_UPDATE) { PyObject *iterable = stack_pointer[-1]; PyObject *set = stack_pointer[-(2 + (oparg-1))];- #line 1497 "Python/bytecodes.c"+ #line 1495 "Python/bytecodes.c" int err = _PySet_Update(set, iterable); #line 2127 "Python/generated_cases.c.h" Py_DECREF(iterable);- #line 1499 "Python/bytecodes.c"+ #line 1497 "Python/bytecodes.c" if (err < 0) goto pop_1_error; #line 2131 "Python/generated_cases.c.h" STACK_SHRINK(1);@@ -2135,7 +2135,7 @@ TARGET(BUILD_SET) { PyObject **values = (stack_pointer - oparg); PyObject *set;- #line 1503 "Python/bytecodes.c"+ #line 1501 "Python/bytecodes.c" set = PySet_New(NULL); if (set == NULL) goto error;@@ -2160,7 +2160,7 @@ TARGET(BUILD_MAP) { PyObject **values = (stack_pointer - oparg*2); PyObject *map;- #line 1520 "Python/bytecodes.c"+ #line 1518 "Python/bytecodes.c" map = _PyDict_FromItems( values, 2, values+1, 2,@@ -2172,7 +2172,7 @@ for (int _i = oparg*2; --_i >= 0;) { Py_DECREF(values[_i]); }- #line 1528 "Python/bytecodes.c"+ #line 1526 "Python/bytecodes.c" if (map == NULL) { STACK_SHRINK(oparg*2); goto error; } #line 2178 "Python/generated_cases.c.h" STACK_SHRINK(oparg*2);@@ -2182,7 +2182,7 @@ } TARGET(SETUP_ANNOTATIONS) {- #line 1532 "Python/bytecodes.c"+ #line 1530 "Python/bytecodes.c" int err; PyObject *ann_dict; if (LOCALS() == NULL) {@@ -2230,7 +2230,7 @@ PyObject *keys = stack_pointer[-1]; PyObject **values = (stack_pointer - (1 + oparg)); PyObject *map;- #line 1574 "Python/bytecodes.c"+ #line 1572 "Python/bytecodes.c" if (!PyTuple_CheckExact(keys) || PyTuple_GET_SIZE(keys) != (Py_ssize_t)oparg) { _PyErr_SetString(tstate, PyExc_SystemError,@@ -2245,7 +2245,7 @@ Py_DECREF(values[_i]); } Py_DECREF(keys);- #line 1584 "Python/bytecodes.c"+ #line 1582 "Python/bytecodes.c" if (map == NULL) { STACK_SHRINK(oparg); goto pop_1_error; } #line 2251 "Python/generated_cases.c.h" STACK_SHRINK(oparg);@@ -2255,7 +2255,7 @@ TARGET(DICT_UPDATE) { PyObject *update = stack_pointer[-1];- #line 1588 "Python/bytecodes.c"+ #line 1586 "Python/bytecodes.c" PyObject *dict = PEEK(oparg + 1); // update is still on the stack if (PyDict_Update(dict, update) < 0) { if (_PyErr_ExceptionMatches(tstate, PyExc_AttributeError)) {@@ -2265,7 +2265,7 @@ } #line 2267 "Python/generated_cases.c.h" Py_DECREF(update);- #line 1596 "Python/bytecodes.c"+ #line 1594 "Python/bytecodes.c" if (true) goto pop_1_error; } #line 2272 "Python/generated_cases.c.h"@@ -2276,14 +2276,14 @@ TARGET(DICT_MERGE) { PyObject *update = stack_pointer[-1];- #line 1602 "Python/bytecodes.c"+ #line 1600 "Python/bytecodes.c" PyObject *dict = PEEK(oparg + 1); // update is still on the stack if (_PyDict_MergeEx(dict, update, 2) < 0) { format_kwargs_error(tstate, PEEK(3 + oparg), update); #line 2285 "Python/generated_cases.c.h" Py_DECREF(update);- #line 1607 "Python/bytecodes.c"+ #line 1605 "Python/bytecodes.c" if (true) goto pop_1_error; } #line 2290 "Python/generated_cases.c.h"@@ -2295,7 +2295,7 @@ TARGET(MAP_ADD) { PyObject *value = stack_pointer[-1]; PyObject *key = stack_pointer[-2];- #line 1613 "Python/bytecodes.c"+ #line 1611 "Python/bytecodes.c" PyObject *dict = PEEK(oparg + 2); // key, value are still on the stack assert(PyDict_CheckExact(dict)); /* dict[key] = value */@@ -2307,7 +2307,7 @@ } TARGET(INSTRUMENTED_LOAD_SUPER_ATTR) {- #line 1621 "Python/bytecodes.c"+ #line 1619 "Python/bytecodes.c" _PySuperAttrCache *cache = (_PySuperAttrCache *)next_instr; // cancel out the decrement that will happen in LOAD_SUPER_ATTR; we // don't want to specialize instrumented instructions@@ -2324,7 +2324,7 @@ PyObject *global_super = stack_pointer[-3]; PyObject *res2 = NULL; PyObject *res;- #line 1635 "Python/bytecodes.c"+ #line 1633 "Python/bytecodes.c" PyObject *name = GETITEM(FRAME_CO_NAMES, oparg >> 2); int load_method = oparg & 1; #if ENABLE_SPECIALIZATION@@ -2370,7 +2370,7 @@ Py_DECREF(global_super); Py_DECREF(class); Py_DECREF(self);- #line 1677 "Python/bytecodes.c"+ #line 1675 "Python/bytecodes.c" if (super == NULL) goto pop_3_error; res = PyObject_GetAttr(super, name); Py_DECREF(super);@@ -2390,7 +2390,7 @@ PyObject *global_super = stack_pointer[-3]; PyObject *res2 = NULL; PyObject *res;- #line 1696 "Python/bytecodes.c"+ #line 1694 "Python/bytecodes.c" assert(!(oparg & 1)); DEOPT_IF(global_super != (PyObject *)&PySuper_Type, LOAD_SUPER_ATTR); DEOPT_IF(!PyType_Check(class), LOAD_SUPER_ATTR);@@ -2401,7 +2401,7 @@ Py_DECREF(global_super); Py_DECREF(class); Py_DECREF(self);- #line 1703 "Python/bytecodes.c"+ #line 1701 "Python/bytecodes.c" if (res == NULL) goto pop_3_error; #line 2407 "Python/generated_cases.c.h" STACK_SHRINK(2);@@ -2418,7 +2418,7 @@ PyObject *global_super = stack_pointer[-3]; PyObject *res2; PyObject *res;- #line 1707 "Python/bytecodes.c"+ #line 1705 "Python/bytecodes.c" assert(oparg & 1); DEOPT_IF(global_super != (PyObject *)&PySuper_Type, LOAD_SUPER_ATTR); DEOPT_IF(!PyType_Check(class), LOAD_SUPER_ATTR);@@ -2455,7 +2455,7 @@ PyObject *owner = stack_pointer[-1]; PyObject *res2 = NULL; PyObject *res;- #line 1746 "Python/bytecodes.c"+ #line 1744 "Python/bytecodes.c" #if ENABLE_SPECIALIZATION _PyAttrCache *cache = (_PyAttrCache *)next_instr; if (ADAPTIVE_COUNTER_IS_ZERO(cache->counter)) {@@ -2491,7 +2491,7 @@ */ #line 2493 "Python/generated_cases.c.h" Py_DECREF(owner);- #line 1780 "Python/bytecodes.c"+ #line 1778 "Python/bytecodes.c" if (meth == NULL) goto pop_1_error; res2 = NULL; res = meth;@@ -2502,7 +2502,7 @@ res = PyObject_GetAttr(owner, name); #line 2504 "Python/generated_cases.c.h" Py_DECREF(owner);- #line 1789 "Python/bytecodes.c"+ #line 1787 "Python/bytecodes.c" if (res == NULL) goto pop_1_error; } #line 2509 "Python/generated_cases.c.h"@@ -2519,7 +2519,7 @@ PyObject *res; uint32_t type_version = read_u32(&next_instr[1].cache); uint16_t index = read_u16(&next_instr[3].cache);- #line 1798 "Python/bytecodes.c"+ #line 1796 "Python/bytecodes.c" PyTypeObject *tp = Py_TYPE(owner); assert(type_version != 0); DEOPT_IF(tp->tp_version_tag != type_version, LOAD_ATTR);@@ -2547,7 +2547,7 @@ PyObject *res; uint32_t type_version = read_u32(&next_instr[1].cache); uint16_t index = read_u16(&next_instr[3].cache);- #line 1814 "Python/bytecodes.c"+ #line 1812 "Python/bytecodes.c" DEOPT_IF(!PyModule_CheckExact(owner), LOAD_ATTR); PyDictObject *dict = (PyDictObject *)((PyModuleObject *)owner)->md_dict; assert(dict != NULL);@@ -2575,7 +2575,7 @@ PyObject *res; uint32_t type_version = read_u32(&next_instr[1].cache); uint16_t index = read_u16(&next_instr[3].cache);- #line 1830 "Python/bytecodes.c"+ #line 1828 "Python/bytecodes.c" PyTypeObject *tp = Py_TYPE(owner); assert(type_version != 0); DEOPT_IF(tp->tp_version_tag != type_version, LOAD_ATTR);@@ -2617,7 +2617,7 @@ PyObject *res; uint32_t type_version = read_u32(&next_instr[1].cache); uint16_t index = read_u16(&next_instr[3].cache);- #line 1860 "Python/bytecodes.c"+ #line 1858 "Python/bytecodes.c" PyTypeObject *tp = Py_TYPE(owner); assert(type_version != 0); DEOPT_IF(tp->tp_version_tag != type_version, LOAD_ATTR);@@ -2642,7 +2642,7 @@ PyObject *res; uint32_t type_version = read_u32(&next_instr[1].cache); PyObject *descr = read_obj(&next_instr[5].cache);- #line 1873 "Python/bytecodes.c"+ #line 1871 "Python/bytecodes.c" DEOPT_IF(!PyType_Check(cls), LOAD_ATTR); DEOPT_IF(((PyTypeObject *)cls)->tp_version_tag != type_version,@@ -2668,7 +2668,7 @@ uint32_t type_version = read_u32(&next_instr[1].cache); uint32_t func_version = read_u32(&next_instr[3].cache); PyObject *fget = read_obj(&next_instr[5].cache);- #line 1888 "Python/bytecodes.c"+ #line 1886 "Python/bytecodes.c" DEOPT_IF(tstate->interp->eval_frame, LOAD_ATTR); PyTypeObject *cls = Py_TYPE(owner);@@ -2700,7 +2700,7 @@ uint32_t type_version = read_u32(&next_instr[1].cache); uint32_t func_version = read_u32(&next_instr[3].cache); PyObject *getattribute = read_obj(&next_instr[5].cache);- #line 1914 "Python/bytecodes.c"+ #line 1912 "Python/bytecodes.c" DEOPT_IF(tstate->interp->eval_frame, LOAD_ATTR); PyTypeObject *cls = Py_TYPE(owner); DEOPT_IF(cls->tp_version_tag != type_version, LOAD_ATTR);@@ -2734,7 +2734,7 @@ PyObject *value = stack_pointer[-2]; uint32_t type_version = read_u32(&next_instr[1].cache); uint16_t index = read_u16(&next_instr[3].cache);- #line 1942 "Python/bytecodes.c"+ #line 1940 "Python/bytecodes.c" PyTypeObject *tp = Py_TYPE(owner); assert(type_version != 0); DEOPT_IF(tp->tp_version_tag != type_version, STORE_ATTR);@@ -2763,7 +2763,7 @@ PyObject *value = stack_pointer[-2]; uint32_t type_version = read_u32(&next_instr[1].cache); uint16_t hint = read_u16(&next_instr[3].cache);- #line 1962 "Python/bytecodes.c"+ #line 1960 "Python/bytecodes.c" PyTypeObject *tp = Py_TYPE(owner); assert(type_version != 0); DEOPT_IF(tp->tp_version_tag != type_version, STORE_ATTR);@@ -2813,7 +2813,7 @@ PyObject *value = stack_pointer[-2]; uint32_t type_version = read_u32(&next_instr[1].cache); uint16_t index = read_u16(&next_instr[3].cache);- #line 2003 "Python/bytecodes.c"+ #line 2001 "Python/bytecodes.c" PyTypeObject *tp = Py_TYPE(owner); assert(type_version != 0); DEOPT_IF(tp->tp_version_tag != type_version, STORE_ATTR);@@ -2835,7 +2835,7 @@ PyObject *right = stack_pointer[-1]; PyObject *left = stack_pointer[-2]; PyObject *res;- #line 2022 "Python/bytecodes.c"+ #line 2020 "Python/bytecodes.c" #if ENABLE_SPECIALIZATION _PyCompareOpCache *cache = (_PyCompareOpCache *)next_instr; if (ADAPTIVE_COUNTER_IS_ZERO(cache->counter)) {@@ -2851,7 +2851,7 @@ #line 2852 "Python/generated_cases.c.h" Py_DECREF(left); Py_DECREF(right);- #line 2035 "Python/bytecodes.c"+ #line 2033 "Python/bytecodes.c" if (res == NULL) goto pop_2_error; #line 2857 "Python/generated_cases.c.h" STACK_SHRINK(1);@@ -2864,7 +2864,7 @@ PyObject *right = stack_pointer[-1]; PyObject *left = stack_pointer[-2]; PyObject *res;- #line 2039 "Python/bytecodes.c"+ #line 2037 "Python/bytecodes.c" DEOPT_IF(!PyFloat_CheckExact(left), COMPARE_OP); DEOPT_IF(!PyFloat_CheckExact(right), COMPARE_OP); STAT_INC(COMPARE_OP, hit);@@ -2886,7 +2886,7 @@ PyObject *right = stack_pointer[-1]; PyObject *left = stack_pointer[-2]; PyObject *res;- #line 2053 "Python/bytecodes.c"+ #line 2051 "Python/bytecodes.c" DEOPT_IF(!PyLong_CheckExact(left), COMPARE_OP); DEOPT_IF(!PyLong_CheckExact(right), COMPARE_OP); DEOPT_IF(!_PyLong_IsCompact((PyLongObject *)left), COMPARE_OP);@@ -2912,7 +2912,7 @@ PyObject *right = stack_pointer[-1]; PyObject *left = stack_pointer[-2]; PyObject *res;- #line 2071 "Python/bytecodes.c"+ #line 2069 "Python/bytecodes.c" DEOPT_IF(!PyUnicode_CheckExact(left), COMPARE_OP); DEOPT_IF(!PyUnicode_CheckExact(right), COMPARE_OP); STAT_INC(COMPARE_OP, hit);@@ -2935,12 +2935,12 @@ PyObject *right = stack_pointer[-1]; PyObject *left = stack_pointer[-2]; PyObject *b;- #line 2085 "Python/bytecodes.c"+ #line 2083 "Python/bytecodes.c" int res = Py_Is(left, right) ^ oparg; #line 2941 "Python/generated_cases.c.h" Py_DECREF(left); Py_DECREF(right);- #line 2087 "Python/bytecodes.c"+ #line 2085 "Python/bytecodes.c" b = res ? Py_True : Py_False; #line 2946 "Python/generated_cases.c.h" STACK_SHRINK(1);@@ -2952,12 +2952,12 @@ PyObject *right = stack_pointer[-1]; PyObject *left = stack_pointer[-2]; PyObject *b;- #line 2091 "Python/bytecodes.c"+ #line 2089 "Python/bytecodes.c" int res = PySequence_Contains(right, left); #line 2958 "Python/generated_cases.c.h" Py_DECREF(left); Py_DECREF(right);- #line 2093 "Python/bytecodes.c"+ #line 2091 "Python/bytecodes.c" if (res < 0) goto pop_2_error; b = (res ^ oparg) ? Py_True : Py_False; #line 2964 "Python/generated_cases.c.h"@@ -2971,12 +2971,12 @@ PyObject *exc_value = stack_pointer[-2]; PyObject *rest; PyObject *match;- #line 2098 "Python/bytecodes.c"+ #line 2096 "Python/bytecodes.c" if (check_except_star_type_valid(tstate, match_type) < 0) { #line 2977 "Python/generated_cases.c.h" Py_DECREF(exc_value); Py_DECREF(match_type);- #line 2100 "Python/bytecodes.c"+ #line 2098 "Python/bytecodes.c" if (true) goto pop_2_error; } @@ -2987,7 +2987,7 @@ #line 2988 "Python/generated_cases.c.h" Py_DECREF(exc_value); Py_DECREF(match_type);- #line 2108 "Python/bytecodes.c"+ #line 2106 "Python/bytecodes.c" if (res < 0) goto pop_2_error; assert((match == NULL) == (rest == NULL));@@ -3006,19 +3006,19 @@ PyObject *right = stack_pointer[-1]; PyObject *left = stack_pointer[-2]; PyObject *b;- #line 2119 "Python/bytecodes.c"+ #line 2117 "Python/bytecodes.c" assert(PyExceptionInstance_Check(left)); if (check_except_type_valid(tstate, right) < 0) { #line 3013 "Python/generated_cases.c.h" Py_DECREF(right);- #line 2122 "Python/bytecodes.c"+ #line 2120 "Python/bytecodes.c" if (true) goto pop_1_error; } int res = PyErr_GivenExceptionMatches(left, right); #line 3020 "Python/generated_cases.c.h" Py_DECREF(right);- #line 2127 "Python/bytecodes.c"+ #line 2125 "Python/bytecodes.c" b = res ? Py_True : Py_False; #line 3024 "Python/generated_cases.c.h" stack_pointer[-1] = b;@@ -3029,13 +3029,13 @@ PyObject *fromlist = stack_pointer[-1]; PyObject *level = stack_pointer[-2]; PyObject *res;- #line 2131 "Python/bytecodes.c"+ #line 2129 "Python/bytecodes.c" PyObject *name = GETITEM(FRAME_CO_NAMES, oparg); res = import_name(tstate, frame, name, fromlist, level); #line 3036 "Python/generated_cases.c.h" Py_DECREF(level); Py_DECREF(fromlist);- #line 2134 "Python/bytecodes.c"+ #line 2132 "Python/bytecodes.c" if (res == NULL) goto pop_2_error; #line 3041 "Python/generated_cases.c.h" STACK_SHRINK(1);@@ -3046,7 +3046,7 @@ TARGET(IMPORT_FROM) { PyObject *from = stack_pointer[-1]; PyObject *res;- #line 2138 "Python/bytecodes.c"+ #line 2136 "Python/bytecodes.c" PyObject *name = GETITEM(FRAME_CO_NAMES, oparg); res = import_from(tstate, from, name); if (res == NULL) goto error;@@ -3057,14 +3057,14 @@ } TARGET(JUMP_FORWARD) {- #line 2144 "Python/bytecodes.c"+ #line 2142 "Python/bytecodes.c" JUMPBY(oparg); #line 3063 "Python/generated_cases.c.h" DISPATCH(); } TARGET(JUMP_BACKWARD) {- #line 2148 "Python/bytecodes.c"+ #line 2146 "Python/bytecodes.c" _Py_CODEUNIT *here = next_instr - 1; assert(oparg <= INSTR_OFFSET()); JUMPBY(1-oparg);@@ -3088,14 +3088,14 @@ } TARGET(ENTER_EXECUTOR) {- #line 2179 "Python/bytecodes.c"+ #line 2177 "Python/bytecodes.c" PyCodeObject *code = _PyFrame_GetCode(frame); _PyExecutorObject *executor = (_PyExecutorObject *)code->co_executors->executors[oparg&255]; Py_INCREF(executor); frame = executor->execute(executor, frame, stack_pointer); if (frame == NULL) { frame = cframe.current_frame;- goto error;+ goto resume_with_error; } goto resume_frame; #line 3102 "Python/generated_cases.c.h"@@ -3103,7 +3103,7 @@ TARGET(POP_JUMP_IF_FALSE) { PyObject *cond = stack_pointer[-1];- #line 2191 "Python/bytecodes.c"+ #line 2189 "Python/bytecodes.c" if (Py_IsFalse(cond)) { JUMPBY(oparg); }@@ -3111,7 +3111,7 @@ int err = PyObject_IsTrue(cond); #line 3113 "Python/generated_cases.c.h" Py_DECREF(cond);- #line 2197 "Python/bytecodes.c"+ #line 2195 "Python/bytecodes.c" if (err == 0) { JUMPBY(oparg); }@@ -3126,7 +3126,7 @@ TARGET(POP_JUMP_IF_TRUE) { PyObject *cond = stack_pointer[-1];- #line 2207 "Python/bytecodes.c"+ #line 2205 "Python/bytecodes.c" if (Py_IsTrue(cond)) { JUMPBY(oparg); }@@ -3134,7 +3134,7 @@ int err = PyObject_IsTrue(cond); #line 3136 "Python/generated_cases.c.h" Py_DECREF(cond);- #line 2213 "Python/bytecodes.c"+ #line 2211 "Python/bytecodes.c" if (err > 0) { JUMPBY(oparg); }@@ -3149,11 +3149,11 @@ TARGET(POP_JUMP_IF_NOT_NONE) { PyObject *value = stack_pointer[-1];- #line 2223 "Python/bytecodes.c"+ #line 2221 "Python/bytecodes.c" if (!Py_IsNone(value)) { #line 3155 "Python/generated_cases.c.h" Py_DECREF(value);- #line 2225 "Python/bytecodes.c"+ #line 2223 "Python/bytecodes.c" JUMPBY(oparg); } #line 3160 "Python/generated_cases.c.h"@@ -3163,14 +3163,14 @@ TARGET(POP_JUMP_IF_NONE) { PyObject *value = stack_pointer[-1];- #line 2230 "Python/bytecodes.c"+ #line 2228 "Python/bytecodes.c" if (Py_IsNone(value)) { JUMPBY(oparg); } else { #line 3172 "Python/generated_cases.c.h" Py_DECREF(value);- #line 2235 "Python/bytecodes.c"+ #line 2233 "Python/bytecodes.c" } #line 3176 "Python/generated_cases.c.h" STACK_SHRINK(1);@@ -3178,7 +3178,7 @@ } TARGET(JUMP_BACKWARD_NO_INTERRUPT) {- #line 2239 "Python/bytecodes.c"+ #line 2237 "Python/bytecodes.c" /* This bytecode is used in the `yield from` or `await` loop. * If there is an interrupt, we want it handled in the innermost * generator or coroutine, so we deliberately do not check it here.@@ -3192,7 +3192,7 @@ TARGET(GET_LEN) { PyObject *obj = stack_pointer[-1]; PyObject *len_o;- #line 2248 "Python/bytecodes.c"+ #line 2246 "Python/bytecodes.c" // PUSH(len(TOS)) Py_ssize_t len_i = PyObject_Length(obj); if (len_i < 0) goto error;@@ -3209,7 +3209,7 @@ PyObject *type = stack_pointer[-2]; PyObject *subject = stack_pointer[-3]; PyObject *attrs;- #line 2256 "Python/bytecodes.c"+ #line 2254 "Python/bytecodes.c" // Pop TOS and TOS1. Set TOS to a tuple of attributes on success, or // None on failure. assert(PyTuple_CheckExact(names));@@ -3218,7 +3218,7 @@ Py_DECREF(subject); Py_DECREF(type); Py_DECREF(names);- #line 2261 "Python/bytecodes.c"+ #line 2259 "Python/bytecodes.c" if (attrs) { assert(PyTuple_CheckExact(attrs)); // Success! }@@ -3235,7 +3235,7 @@ TARGET(MATCH_MAPPING) { PyObject *subject = stack_pointer[-1]; PyObject *res;- #line 2271 "Python/bytecodes.c"+ #line 2269 "Python/bytecodes.c" int match = Py_TYPE(subject)->tp_flags & Py_TPFLAGS_MAPPING; res = match ? Py_True : Py_False; #line 3242 "Python/generated_cases.c.h"@@ -3247,7 +3247,7 @@ TARGET(MATCH_SEQUENCE) { PyObject *subject = stack_pointer[-1]; PyObject *res;- #line 2276 "Python/bytecodes.c"+ #line 2274 "Python/bytecodes.c" int match = Py_TYPE(subject)->tp_flags & Py_TPFLAGS_SEQUENCE; res = match ? Py_True : Py_False; #line 3254 "Python/generated_cases.c.h"@@ -3260,7 +3260,7 @@ PyObject *keys = stack_pointer[-1]; PyObject *subject = stack_pointer[-2]; PyObject *values_or_none;- #line 2281 "Python/bytecodes.c"+ #line 2279 "Python/bytecodes.c" // On successful match, PUSH(values). Otherwise, PUSH(None). values_or_none = match_keys(tstate, subject, keys); if (values_or_none == NULL) goto error;@@ -3273,12 +3273,12 @@ TARGET(GET_ITER) { PyObject *iterable = stack_pointer[-1]; PyObject *iter;- #line 2287 "Python/bytecodes.c"+ #line 2285 "Python/bytecodes.c" /* before: [obj]; after [getiter(obj)] */ iter = PyObject_GetIter(iterable); #line 3280 "Python/generated_cases.c.h" Py_DECREF(iterable);- #line 2290 "Python/bytecodes.c"+ #line 2288 "Python/bytecodes.c" if (iter == NULL) goto pop_1_error; #line 3284 "Python/generated_cases.c.h" stack_pointer[-1] = iter;@@ -3288,7 +3288,7 @@ TARGET(GET_YIELD_FROM_ITER) { PyObject *iterable = stack_pointer[-1]; PyObject *iter;- #line 2294 "Python/bytecodes.c"+ #line 2292 "Python/bytecodes.c" /* before: [obj]; after [getiter(obj)] */ if (PyCoro_CheckExact(iterable)) { /* `iterable` is a coroutine */@@ -3313,7 +3313,7 @@ } #line 3315 "Python/generated_cases.c.h" Py_DECREF(iterable);- #line 2317 "Python/bytecodes.c"+ #line 2315 "Python/bytecodes.c" } #line 3319 "Python/generated_cases.c.h" stack_pointer[-1] = iter;@@ -3325,7 +3325,7 @@ static_assert(INLINE_CACHE_ENTRIES_FOR_ITER == 1, "incorrect cache size"); PyObject *iter = stack_pointer[-1]; PyObject *next;- #line 2335 "Python/bytecodes.c"+ #line 2333 "Python/bytecodes.c" #if ENABLE_SPECIALIZATION _PyForIterCache *cache = (_PyForIterCache *)next_instr; if (ADAPTIVE_COUNTER_IS_ZERO(cache->counter)) {@@ -3365,7 +3365,7 @@ } TARGET(INSTRUMENTED_FOR_ITER) {- #line 2369 "Python/bytecodes.c"+ #line 2367 "Python/bytecodes.c" _Py_CODEUNIT *here = next_instr-1; _Py_CODEUNIT *target; PyObject *iter = TOP();@@ -3398,7 +3398,7 @@ TARGET(FOR_ITER_LIST) { PyObject *iter = stack_pointer[-1]; PyObject *next;- #line 2397 "Python/bytecodes.c"+ #line 2395 "Python/bytecodes.c" DEOPT_IF(Py_TYPE(iter) != &PyListIter_Type, FOR_ITER); _PyListIterObject *it = (_PyListIterObject *)iter; STAT_INC(FOR_ITER, hit);@@ -3429,7 +3429,7 @@ TARGET(FOR_ITER_TUPLE) { PyObject *iter = stack_pointer[-1]; PyObject *next;- #line 2420 "Python/bytecodes.c"+ #line 2418 "Python/bytecodes.c" _PyTupleIterObject *it = (_PyTupleIterObject *)iter; DEOPT_IF(Py_TYPE(it) != &PyTupleIter_Type, FOR_ITER); STAT_INC(FOR_ITER, hit);@@ -3460,7 +3460,7 @@ TARGET(FOR_ITER_RANGE) { PyObject *iter = stack_pointer[-1]; PyObject *next;- #line 2443 "Python/bytecodes.c"+ #line 2441 "Python/bytecodes.c" _PyRangeIterObject *r = (_PyRangeIterObject *)iter; DEOPT_IF(Py_TYPE(r) != &PyRangeIter_Type, FOR_ITER); STAT_INC(FOR_ITER, hit);@@ -3488,7 +3488,7 @@ TARGET(FOR_ITER_GEN) { PyObject *iter = stack_pointer[-1];- #line 2464 "Python/bytecodes.c"+ #line 2462 "Python/bytecodes.c" DEOPT_IF(tstate->interp->eval_frame, FOR_ITER); PyGenObject *gen = (PyGenObject *)iter; DEOPT_IF(Py_TYPE(gen) != &PyGen_Type, FOR_ITER);@@ -3511,7 +3511,7 @@ PyObject *mgr = stack_pointer[-1]; PyObject *exit; PyObject *res;- #line 2482 "Python/bytecodes.c"+ #line 2480 "Python/bytecodes.c" PyObject *enter = _PyObject_LookupSpecial(mgr, &_Py_ID(__aenter__)); if (enter == NULL) { if (!_PyErr_Occurred(tstate)) {@@ -3536,7 +3536,7 @@ } #line 3538 "Python/generated_cases.c.h" Py_DECREF(mgr);- #line 2505 "Python/bytecodes.c"+ #line 2503 "Python/bytecodes.c" res = _PyObject_CallNoArgs(enter); Py_DECREF(enter); if (res == NULL) {@@ -3554,7 +3554,7 @@ PyObject *mgr = stack_pointer[-1]; PyObject *exit; PyObject *res;- #line 2514 "Python/bytecodes.c"+ #line 2512 "Python/bytecodes.c" /* pop the context manager, push its __exit__ and the * value returned from calling its __enter__ */@@ -3582,7 +3582,7 @@ } #line 3584 "Python/generated_cases.c.h" Py_DECREF(mgr);- #line 2540 "Python/bytecodes.c"+ #line 2538 "Python/bytecodes.c" res = _PyObject_CallNoArgs(enter); Py_DECREF(enter); if (res == NULL) {@@ -3601,7 +3601,7 @@ PyObject *lasti = stack_pointer[-3]; PyObject *exit_func = stack_pointer[-4]; PyObject *res;- #line 2549 "Python/bytecodes.c"+ #line 2547 "Python/bytecodes.c" /* At the top of the stack are 4 values: - val: TOP = exc_info() - unused: SECOND = previous exception@@ -3631,7 +3631,7 @@ TARGET(PUSH_EXC_INFO) { PyObject *new_exc = stack_pointer[-1]; PyObject *prev_exc;- #line 2588 "Python/bytecodes.c"+ #line 2586 "Python/bytecodes.c" _PyErr_StackItem *exc_info = tstate->exc_info; if (exc_info->exc_value != NULL) { prev_exc = exc_info->exc_value;@@ -3655,7 +3655,7 @@ uint32_t type_version = read_u32(&next_instr[1].cache); uint32_t keys_version = read_u32(&next_instr[3].cache); PyObject *descr = read_obj(&next_instr[5].cache);- #line 2600 "Python/bytecodes.c"+ #line 2598 "Python/bytecodes.c" /* Cached method object */ PyTypeObject *self_cls = Py_TYPE(self); assert(type_version != 0);@@ -3686,7 +3686,7 @@ PyObject *res; uint32_t type_version = read_u32(&next_instr[1].cache); PyObject *descr = read_obj(&next_instr[5].cache);- #line 2619 "Python/bytecodes.c"+ #line 2617 "Python/bytecodes.c" PyTypeObject *self_cls = Py_TYPE(self); DEOPT_IF(self_cls->tp_version_tag != type_version, LOAD_ATTR); assert(self_cls->tp_dictoffset == 0);@@ -3710,7 +3710,7 @@ PyObject *res; uint32_t type_version = read_u32(&next_instr[1].cache); PyObject *descr = read_obj(&next_instr[5].cache);- #line 2631 "Python/bytecodes.c"+ #line 2629 "Python/bytecodes.c" PyTypeObject *self_cls = Py_TYPE(self); DEOPT_IF(self_cls->tp_version_tag != type_version, LOAD_ATTR); Py_ssize_t dictoffset = self_cls->tp_dictoffset;@@ -3733,7 +3733,7 @@ } TARGET(KW_NAMES) {- #line 2647 "Python/bytecodes.c"+ #line 2645 "Python/bytecodes.c" assert(kwnames == NULL); assert(oparg < PyTuple_GET_SIZE(FRAME_CO_CONSTS)); kwnames = GETITEM(FRAME_CO_CONSTS, oparg);@@ -3742,7 +3742,7 @@ } TARGET(INSTRUMENTED_CALL) {- #line 2653 "Python/bytecodes.c"+ #line 2651 "Python/bytecodes.c" int is_meth = PEEK(oparg+2) != NULL; int total_args = oparg + is_meth; PyObject *function = PEEK(total_args + 1);@@ -3765,7 +3765,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res;- #line 2699 "Python/bytecodes.c"+ #line 2697 "Python/bytecodes.c" int is_meth = method != NULL; int total_args = oparg; if (is_meth) {@@ -3859,7 +3859,7 @@ TARGET(CALL_BOUND_METHOD_EXACT_ARGS) { PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)];- #line 2787 "Python/bytecodes.c"+ #line 2785 "Python/bytecodes.c" DEOPT_IF(method != NULL, CALL); DEOPT_IF(Py_TYPE(callable) != &PyMethod_Type, CALL); STAT_INC(CALL, hit);@@ -3878,7 +3878,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; uint32_t func_version = read_u32(&next_instr[1].cache);- #line 2799 "Python/bytecodes.c"+ #line 2797 "Python/bytecodes.c" assert(kwnames == NULL); DEOPT_IF(tstate->interp->eval_frame, CALL); int is_meth = method != NULL;@@ -3912,7 +3912,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; uint32_t func_version = read_u32(&next_instr[1].cache);- #line 2827 "Python/bytecodes.c"+ #line 2825 "Python/bytecodes.c" assert(kwnames == NULL); DEOPT_IF(tstate->interp->eval_frame, CALL); int is_meth = method != NULL;@@ -3956,7 +3956,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *null = stack_pointer[-(2 + oparg)]; PyObject *res;- #line 2865 "Python/bytecodes.c"+ #line 2863 "Python/bytecodes.c" assert(kwnames == NULL); assert(oparg == 1); DEOPT_IF(null != NULL, CALL);@@ -3979,7 +3979,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *null = stack_pointer[-(2 + oparg)]; PyObject *res;- #line 2877 "Python/bytecodes.c"+ #line 2875 "Python/bytecodes.c" assert(kwnames == NULL); assert(oparg == 1); DEOPT_IF(null != NULL, CALL);@@ -4004,7 +4004,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *null = stack_pointer[-(2 + oparg)]; PyObject *res;- #line 2891 "Python/bytecodes.c"+ #line 2889 "Python/bytecodes.c" assert(kwnames == NULL); assert(oparg == 1); DEOPT_IF(null != NULL, CALL);@@ -4028,7 +4028,7 @@ PyObject **args = (stack_pointer - oparg); PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *null = stack_pointer[-(2 + oparg)];- #line 2905 "Python/bytecodes.c"+ #line 2903 "Python/bytecodes.c" /* This instruction does the following: * 1. Creates the object (by calling ``object.__new__``) * 2. Pushes a shim frame to the frame stack (to cleanup after ``__init__``)@@ -4083,7 +4083,7 @@ TARGET(EXIT_INIT_CHECK) { PyObject *should_be_none = stack_pointer[-1];- #line 2957 "Python/bytecodes.c"+ #line 2955 "Python/bytecodes.c" assert(STACK_LEVEL() == 2); if (should_be_none != Py_None) { PyErr_Format(PyExc_TypeError,@@ -4101,7 +4101,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res;- #line 2967 "Python/bytecodes.c"+ #line 2965 "Python/bytecodes.c" int is_meth = method != NULL; int total_args = oparg; if (is_meth) {@@ -4137,7 +4137,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res;- #line 2992 "Python/bytecodes.c"+ #line 2990 "Python/bytecodes.c" /* Builtin METH_O functions */ assert(kwnames == NULL); int is_meth = method != NULL;@@ -4179,7 +4179,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res;- #line 3023 "Python/bytecodes.c"+ #line 3021 "Python/bytecodes.c" /* Builtin METH_FASTCALL functions, without keywords */ assert(kwnames == NULL); int is_meth = method != NULL;@@ -4225,7 +4225,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res;- #line 3058 "Python/bytecodes.c"+ #line 3056 "Python/bytecodes.c" /* Builtin METH_FASTCALL | METH_KEYWORDS functions */ int is_meth = method != NULL; int total_args = oparg;@@ -4271,7 +4271,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res;- #line 3093 "Python/bytecodes.c"+ #line 3091 "Python/bytecodes.c" assert(kwnames == NULL); /* len(o) */ int is_meth = method != NULL;@@ -4309,7 +4309,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res;- #line 3120 "Python/bytecodes.c"+ #line 3118 "Python/bytecodes.c" assert(kwnames == NULL); /* isinstance(o, o2) */ int is_meth = method != NULL;@@ -4348,7 +4348,7 @@ PyObject **args = (stack_pointer - oparg); PyObject *self = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)];- #line 3150 "Python/bytecodes.c"+ #line 3148 "Python/bytecodes.c" assert(kwnames == NULL); assert(oparg == 1); assert(method != NULL);@@ -4373,7 +4373,7 @@ PyObject **args = (stack_pointer - oparg); PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res;- #line 3170 "Python/bytecodes.c"+ #line 3168 "Python/bytecodes.c" assert(kwnames == NULL); int is_meth = method != NULL; int total_args = oparg;@@ -4417,7 +4417,7 @@ PyObject **args = (stack_pointer - oparg); PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res;- #line 3204 "Python/bytecodes.c"+ #line 3202 "Python/bytecodes.c" int is_meth = method != NULL; int total_args = oparg; if (is_meth) {@@ -4459,7 +4459,7 @@ PyObject **args = (stack_pointer - oparg); PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res;- #line 3236 "Python/bytecodes.c"+ #line 3234 "Python/bytecodes.c" assert(kwnames == NULL); assert(oparg == 0 || oparg == 1); int is_meth = method != NULL;@@ -4501,7 +4501,7 @@ PyObject **args = (stack_pointer - oparg); PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res;- #line 3268 "Python/bytecodes.c"+ #line 3266 "Python/bytecodes.c" assert(kwnames == NULL); int is_meth = method != NULL; int total_args = oparg;@@ -4539,7 +4539,7 @@ } TARGET(INSTRUMENTED_CALL_FUNCTION_EX) {- #line 3299 "Python/bytecodes.c"+ #line 3297 "Python/bytecodes.c" GO_TO_INSTRUCTION(CALL_FUNCTION_EX); #line 4545 "Python/generated_cases.c.h" }@@ -4550,7 +4550,7 @@ PyObject *callargs = stack_pointer[-(1 + ((oparg & 1) ? 1 : 0))]; PyObject *func = stack_pointer[-(2 + ((oparg & 1) ? 1 : 0))]; PyObject *result;- #line 3303 "Python/bytecodes.c"+ #line 3301 "Python/bytecodes.c" // DICT_MERGE is called before this opcode if there are kwargs. // It converts all dict subtypes in kwargs into regular dicts. assert(kwargs == NULL || PyDict_CheckExact(kwargs));@@ -4616,7 +4616,7 @@ Py_DECREF(func); Py_DECREF(callargs); Py_XDECREF(kwargs);- #line 3365 "Python/bytecodes.c"+ #line 3363 "Python/bytecodes.c" assert(PEEK(3 + (oparg & 1)) == NULL); if (result == NULL) { STACK_SHRINK(((oparg & 1) ? 1 : 0)); goto pop_3_error; } #line 4623 "Python/generated_cases.c.h"@@ -4630,7 +4630,7 @@ TARGET(MAKE_FUNCTION) { PyObject *codeobj = stack_pointer[-1]; PyObject *func;- #line 3371 "Python/bytecodes.c"+ #line 3369 "Python/bytecodes.c" PyFunctionObject *func_obj = (PyFunctionObject *) PyFunction_New(codeobj, GLOBALS());@@ -4650,7 +4650,7 @@ TARGET(SET_FUNCTION_ATTRIBUTE) { PyObject *func = stack_pointer[-1]; PyObject *attr = stack_pointer[-2];- #line 3385 "Python/bytecodes.c"+ #line 3383 "Python/bytecodes.c" assert(PyFunction_Check(func)); PyFunctionObject *func_obj = (PyFunctionObject *)func; switch(oparg) {@@ -4682,7 +4682,7 @@ } TARGET(RETURN_GENERATOR) {- #line 3412 "Python/bytecodes.c"+ #line 3410 "Python/bytecodes.c" assert(PyFunction_Check(frame->f_funcobj)); PyFunctionObject *func = (PyFunctionObject *)frame->f_funcobj; PyGenObject *gen = (PyGenObject *)_Py_MakeCoro(func);@@ -4711,13 +4711,13 @@ PyObject *stop = stack_pointer[-(1 + ((oparg == 3) ? 1 : 0))]; PyObject *start = stack_pointer[-(2 + ((oparg == 3) ? 1 : 0))]; PyObject *slice;- #line 3435 "Python/bytecodes.c"+ #line 3433 "Python/bytecodes.c" slice = PySlice_New(start, stop, step); #line 4717 "Python/generated_cases.c.h" Py_DECREF(start); Py_DECREF(stop); Py_XDECREF(step);- #line 3437 "Python/bytecodes.c"+ #line 3435 "Python/bytecodes.c" if (slice == NULL) { STACK_SHRINK(((oparg == 3) ? 1 : 0)); goto pop_2_error; } #line 4723 "Python/generated_cases.c.h" STACK_SHRINK(((oparg == 3) ? 1 : 0));@@ -4729,7 +4729,7 @@ TARGET(CONVERT_VALUE) { PyObject *value = stack_pointer[-1]; PyObject *result;- #line 3441 "Python/bytecodes.c"+ #line 3439 "Python/bytecodes.c" convertion_func_ptr conv_fn; assert(oparg >= FVC_STR && oparg <= FVC_ASCII); conv_fn = CONVERSION_FUNCTIONS[oparg];@@ -4744,7 +4744,7 @@ TARGET(FORMAT_SIMPLE) { PyObject *value = stack_pointer[-1]; PyObject *res;- #line 3450 "Python/bytecodes.c"+ #line 3448 "Python/bytecodes.c" /* If value is a unicode object, then we know the result * of format(value) is value itself. */ if (!PyUnicode_CheckExact(value)) {@@ -4764,7 +4764,7 @@ PyObject *fmt_spec = stack_pointer[-1]; PyObject *value = stack_pointer[-2]; PyObject *res;- #line 3463 "Python/bytecodes.c"+ #line 3461 "Python/bytecodes.c" res = PyObject_Format(value, fmt_spec); Py_DECREF(value); Py_DECREF(fmt_spec);@@ -4778,7 +4778,7 @@ TARGET(COPY) { PyObject *bottom = stack_pointer[-(1 + (oparg-1))]; PyObject *top;- #line 3470 "Python/bytecodes.c"+ #line 3468 "Python/bytecodes.c" assert(oparg > 0); top = Py_NewRef(bottom); #line 4785 "Python/generated_cases.c.h"@@ -4793,7 +4793,7 @@ PyObject *rhs = stack_pointer[-1]; PyObject *lhs = stack_pointer[-2]; PyObject *res;- #line 3475 "Python/bytecodes.c"+ #line 3473 "Python/bytecodes.c" #if ENABLE_SPECIALIZATION _PyBinaryOpCache *cache = (_PyBinaryOpCache *)next_instr; if (ADAPTIVE_COUNTER_IS_ZERO(cache->counter)) {@@ -4811,7 +4811,7 @@ #line 4812 "Python/generated_cases.c.h" Py_DECREF(lhs); Py_DECREF(rhs);- #line 3490 "Python/bytecodes.c"+ #line 3488 "Python/bytecodes.c" if (res == NULL) goto pop_2_error; #line 4817 "Python/generated_cases.c.h" STACK_SHRINK(1);@@ -4823,7 +4823,7 @@ TARGET(SWAP) { PyObject *top = stack_pointer[-1]; PyObject *bottom = stack_pointer[-(2 + (oparg-2))];- #line 3495 "Python/bytecodes.c"+ #line 3493 "Python/bytecodes.c" assert(oparg >= 2); #line 4829 "Python/generated_cases.c.h" stack_pointer[-1] = bottom;@@ -4832,7 +4832,7 @@ } TARGET(INSTRUMENTED_INSTRUCTION) {- #line 3499 "Python/bytecodes.c"+ #line 3497 "Python/bytecodes.c" int next_opcode = _Py_call_instrumentation_instruction( tstate, frame, next_instr-1); if (next_opcode < 0) goto error;@@ -4848,14 +4848,14 @@ } TARGET(INSTRUMENTED_JUMP_FORWARD) {- #line 3513 "Python/bytecodes.c"+ #line 3511 "Python/bytecodes.c" INSTRUMENTED_JUMP(next_instr-1, next_instr+oparg, PY_MONITORING_EVENT_JUMP); #line 4854 "Python/generated_cases.c.h" DISPATCH(); } TARGET(INSTRUMENTED_JUMP_BACKWARD) {- #line 3517 "Python/bytecodes.c"+ #line 3515 "Python/bytecodes.c" INSTRUMENTED_JUMP(next_instr-1, next_instr+1-oparg, PY_MONITORING_EVENT_JUMP); #line 4861 "Python/generated_cases.c.h" CHECK_EVAL_BREAKER();@@ -4863,7 +4863,7 @@ } TARGET(INSTRUMENTED_POP_JUMP_IF_TRUE) {- #line 3522 "Python/bytecodes.c"+ #line 3520 "Python/bytecodes.c" PyObject *cond = POP(); int err = PyObject_IsTrue(cond); Py_DECREF(cond);@@ -4877,7 +4877,7 @@ } TARGET(INSTRUMENTED_POP_JUMP_IF_FALSE) {- #line 3533 "Python/bytecodes.c"+ #line 3531 "Python/bytecodes.c" PyObject *cond = POP(); int err = PyObject_IsTrue(cond); Py_DECREF(cond);@@ -4891,7 +4891,7 @@ } TARGET(INSTRUMENTED_POP_JUMP_IF_NONE) {- #line 3544 "Python/bytecodes.c"+ #line 3542 "Python/bytecodes.c" PyObject *value = POP(); _Py_CODEUNIT *here = next_instr-1; int offset;@@ -4908,7 +4908,7 @@ } TARGET(INSTRUMENTED_POP_JUMP_IF_NOT_NONE) {- #line 3558 "Python/bytecodes.c"+ #line 3556 "Python/bytecodes.c" PyObject *value = POP(); _Py_CODEUNIT *here = next_instr-1; int offset;@@ -4925,7 +4925,7 @@ } TARGET(EXTENDED_ARG) {- #line 3572 "Python/bytecodes.c"+ #line 3570 "Python/bytecodes.c" assert(oparg); opcode = next_instr->op.code; oparg = oparg << 8 | next_instr->op.arg;@@ -4935,14 +4935,14 @@ } TARGET(CACHE) {- #line 3580 "Python/bytecodes.c"+ #line 3578 "Python/bytecodes.c" assert(0 && "Executing a cache."); Py_UNREACHABLE(); #line 4942 "Python/generated_cases.c.h" } TARGET(RESERVED) {- #line 3585 "Python/bytecodes.c"+ #line 3583 "Python/bytecodes.c" assert(0 && "Executing RESERVED instruction."); Py_UNREACHABLE(); #line 4949 "Python/generated_cases.c.h"diff --git a/Python/opcode_metadata.h b/Python/opcode_metadata.hindex 421d9795e7da7..4e31a5e69614b 100644--- a/Python/opcode_metadata.h+++ b/Python/opcode_metadata.h@@ -18,6 +18,21 @@ ((OP) == POP_BLOCK) || \ 0 +#define EXIT_TRACE 300+#define SET_IP 301+#define _GUARD_BOTH_INT 302+#define _BINARY_OP_MULTIPLY_INT 303+#define _BINARY_OP_ADD_INT 304+#define _BINARY_OP_SUBTRACT_INT 305+#define _GUARD_BOTH_FLOAT 306+#define _BINARY_OP_MULTIPLY_FLOAT 307+#define _BINARY_OP_ADD_FLOAT 308+#define _BINARY_OP_SUBTRACT_FLOAT 309+#define _GUARD_BOTH_UNICODE 310+#define _BINARY_OP_ADD_UNICODE 311+#define _LOAD_LOCALS 312+#define _LOAD_FROM_DICT_OR_GLOBALS 313+ #ifndef NEED_OPCODE_METADATA extern int _PyOpcode_num_popped(int opcode, int oparg, bool jump); #else@@ -885,12 +900,19 @@ struct opcode_metadata { int flags; }; +struct opcode_macro_expansion {+ int nuops;+ struct { int16_t uop; int8_t size; int8_t offset; } uops[8];+};++ #define OPCODE_METADATA_FMT(OP) (_PyOpcode_opcode_metadata[(OP)].instr_format) #define SAME_OPCODE_METADATA(OP1, OP2) \ (OPCODE_METADATA_FMT(OP1) == OPCODE_METADATA_FMT(OP2)) #ifndef NEED_OPCODE_METADATA extern const struct opcode_metadata _PyOpcode_opcode_metadata[512];+extern const struct opcode_macro_expansion _PyOpcode_macro_expansion[256]; #else const struct opcode_metadata _PyOpcode_opcode_metadata[512] = { [NOP] = { true, INSTR_FMT_IX, 0 },@@ -1101,4 +1123,88 @@ const struct opcode_metadata _PyOpcode_opcode_metadata[512] = { [CACHE] = { true, INSTR_FMT_IX, 0 }, [RESERVED] = { true, INSTR_FMT_IX, 0 }, };+const struct opcode_macro_expansion _PyOpcode_macro_expansion[256] = {+ [NOP] = { .nuops = 1, .uops = { { NOP, 0, 0 } } },+ [LOAD_FAST] = { .nuops = 1, .uops = { { LOAD_FAST, 0, 0 } } },+ [LOAD_FAST_AND_CLEAR] = { .nuops = 1, .uops = { { LOAD_FAST_AND_CLEAR, 0, 0 } } },+ [LOAD_CONST] = { .nuops = 1, .uops = { { LOAD_CONST, 0, 0 } } },+ [STORE_FAST] = { .nuops = 1, .uops = { { STORE_FAST, 0, 0 } } },+ [POP_TOP] = { .nuops = 1, .uops = { { POP_TOP, 0, 0 } } },+ [PUSH_NULL] = { .nuops = 1, .uops = { { PUSH_NULL, 0, 0 } } },+ [END_SEND] = { .nuops = 1, .uops = { { END_SEND, 0, 0 } } },+ [UNARY_NEGATIVE] = { .nuops = 1, .uops = { { UNARY_NEGATIVE, 0, 0 } } },+ [UNARY_NOT] = { .nuops = 1, .uops = { { UNARY_NOT, 0, 0 } } },+ [UNARY_INVERT] = { .nuops = 1, .uops = { { UNARY_INVERT, 0, 0 } } },+ [BINARY_SLICE] = { .nuops = 1, .uops = { { BINARY_SLICE, 0, 0 } } },+ [STORE_SLICE] = { .nuops = 1, .uops = { { STORE_SLICE, 0, 0 } } },+ [BINARY_SUBSCR_LIST_INT] = { .nuops = 1, .uops = { { BINARY_SUBSCR_LIST_INT, 0, 0 } } },+ [BINARY_SUBSCR_TUPLE_INT] = { .nuops = 1, .uops = { { BINARY_SUBSCR_TUPLE_INT, 0, 0 } } },+ [BINARY_SUBSCR_DICT] = { .nuops = 1, .uops = { { BINARY_SUBSCR_DICT, 0, 0 } } },+ [LIST_APPEND] = { .nuops = 1, .uops = { { LIST_APPEND, 0, 0 } } },+ [SET_ADD] = { .nuops = 1, .uops = { { SET_ADD, 0, 0 } } },+ [STORE_SUBSCR_LIST_INT] = { .nuops = 1, .uops = { { STORE_SUBSCR_LIST_INT, 0, 0 } } },+ [STORE_SUBSCR_DICT] = { .nuops = 1, .uops = { { STORE_SUBSCR_DICT, 0, 0 } } },+ [DELETE_SUBSCR] = { .nuops = 1, .uops = { { DELETE_SUBSCR, 0, 0 } } },+ [CALL_INTRINSIC_1] = { .nuops = 1, .uops = { { CALL_INTRINSIC_1, 0, 0 } } },+ [CALL_INTRINSIC_2] = { .nuops = 1, .uops = { { CALL_INTRINSIC_2, 0, 0 } } },+ [GET_AITER] = { .nuops = 1, .uops = { { GET_AITER, 0, 0 } } },+ [GET_ANEXT] = { .nuops = 1, .uops = { { GET_ANEXT, 0, 0 } } },+ [GET_AWAITABLE] = { .nuops = 1, .uops = { { GET_AWAITABLE, 0, 0 } } },+ [POP_EXCEPT] = { .nuops = 1, .uops = { { POP_EXCEPT, 0, 0 } } },+ [LOAD_ASSERTION_ERROR] = { .nuops = 1, .uops = { { LOAD_ASSERTION_ERROR, 0, 0 } } },+ [LOAD_BUILD_CLASS] = { .nuops = 1, .uops = { { LOAD_BUILD_CLASS, 0, 0 } } },+ [STORE_NAME] = { .nuops = 1, .uops = { { STORE_NAME, 0, 0 } } },+ [DELETE_NAME] = { .nuops = 1, .uops = { { DELETE_NAME, 0, 0 } } },+ [UNPACK_SEQUENCE_TWO_TUPLE] = { .nuops = 1, .uops = { { UNPACK_SEQUENCE_TWO_TUPLE, 0, 0 } } },+ [UNPACK_SEQUENCE_TUPLE] = { .nuops = 1, .uops = { { UNPACK_SEQUENCE_TUPLE, 0, 0 } } },+ [UNPACK_SEQUENCE_LIST] = { .nuops = 1, .uops = { { UNPACK_SEQUENCE_LIST, 0, 0 } } },+ [UNPACK_EX] = { .nuops = 1, .uops = { { UNPACK_EX, 0, 0 } } },+ [DELETE_ATTR] = { .nuops = 1, .uops = { { DELETE_ATTR, 0, 0 } } },+ [STORE_GLOBAL] = { .nuops = 1, .uops = { { STORE_GLOBAL, 0, 0 } } },+ [DELETE_GLOBAL] = { .nuops = 1, .uops = { { DELETE_GLOBAL, 0, 0 } } },+ [DELETE_DEREF] = { .nuops = 1, .uops = { { DELETE_DEREF, 0, 0 } } },+ [LOAD_FROM_DICT_OR_DEREF] = { .nuops = 1, .uops = { { LOAD_FROM_DICT_OR_DEREF, 0, 0 } } },+ [LOAD_DEREF] = { .nuops = 1, .uops = { { LOAD_DEREF, 0, 0 } } },+ [STORE_DEREF] = { .nuops = 1, .uops = { { STORE_DEREF, 0, 0 } } },+ [COPY_FREE_VARS] = { .nuops = 1, .uops = { { COPY_FREE_VARS, 0, 0 } } },+ [BUILD_STRING] = { .nuops = 1, .uops = { { BUILD_STRING, 0, 0 } } },+ [BUILD_TUPLE] = { .nuops = 1, .uops = { { BUILD_TUPLE, 0, 0 } } },+ [BUILD_LIST] = { .nuops = 1, .uops = { { BUILD_LIST, 0, 0 } } },+ [LIST_EXTEND] = { .nuops = 1, .uops = { { LIST_EXTEND, 0, 0 } } },+ [SET_UPDATE] = { .nuops = 1, .uops = { { SET_UPDATE, 0, 0 } } },+ [BUILD_SET] = { .nuops = 1, .uops = { { BUILD_SET, 0, 0 } } },+ [BUILD_MAP] = { .nuops = 1, .uops = { { BUILD_MAP, 0, 0 } } },+ [SETUP_ANNOTATIONS] = { .nuops = 1, .uops = { { SETUP_ANNOTATIONS, 0, 0 } } },+ [BUILD_CONST_KEY_MAP] = { .nuops = 1, .uops = { { BUILD_CONST_KEY_MAP, 0, 0 } } },+ [DICT_UPDATE] = { .nuops = 1, .uops = { { DICT_UPDATE, 0, 0 } } },+ [DICT_MERGE] = { .nuops = 1, .uops = { { DICT_MERGE, 0, 0 } } },+ [MAP_ADD] = { .nuops = 1, .uops = { { MAP_ADD, 0, 0 } } },+ [LOAD_SUPER_ATTR_ATTR] = { .nuops = 1, .uops = { { LOAD_SUPER_ATTR_ATTR, 0, 0 } } },+ [LOAD_SUPER_ATTR_METHOD] = { .nuops = 1, .uops = { { LOAD_SUPER_ATTR_METHOD, 0, 0 } } },+ [COMPARE_OP_FLOAT] = { .nuops = 1, .uops = { { COMPARE_OP_FLOAT, 0, 0 } } },+ [COMPARE_OP_INT] = { .nuops = 1, .uops = { { COMPARE_OP_INT, 0, 0 } } },+ [COMPARE_OP_STR] = { .nuops = 1, .uops = { { COMPARE_OP_STR, 0, 0 } } },+ [IS_OP] = { .nuops = 1, .uops = { { IS_OP, 0, 0 } } },+ [CONTAINS_OP] = { .nuops = 1, .uops = { { CONTAINS_OP, 0, 0 } } },+ [CHECK_EG_MATCH] = { .nuops = 1, .uops = { { CHECK_EG_MATCH, 0, 0 } } },+ [CHECK_EXC_MATCH] = { .nuops = 1, .uops = { { CHECK_EXC_MATCH, 0, 0 } } },+ [GET_LEN] = { .nuops = 1, .uops = { { GET_LEN, 0, 0 } } },+ [MATCH_CLASS] = { .nuops = 1, .uops = { { MATCH_CLASS, 0, 0 } } },+ [MATCH_MAPPING] = { .nuops = 1, .uops = { { MATCH_MAPPING, 0, 0 } } },+ [MATCH_SEQUENCE] = { .nuops = 1, .uops = { { MATCH_SEQUENCE, 0, 0 } } },+ [MATCH_KEYS] = { .nuops = 1, .uops = { { MATCH_KEYS, 0, 0 } } },+ [GET_ITER] = { .nuops = 1, .uops = { { GET_ITER, 0, 0 } } },+ [GET_YIELD_FROM_ITER] = { .nuops = 1, .uops = { { GET_YIELD_FROM_ITER, 0, 0 } } },+ [WITH_EXCEPT_START] = { .nuops = 1, .uops = { { WITH_EXCEPT_START, 0, 0 } } },+ [PUSH_EXC_INFO] = { .nuops = 1, .uops = { { PUSH_EXC_INFO, 0, 0 } } },+ [EXIT_INIT_CHECK] = { .nuops = 1, .uops = { { EXIT_INIT_CHECK, 0, 0 } } },+ [MAKE_FUNCTION] = { .nuops = 1, .uops = { { MAKE_FUNCTION, 0, 0 } } },+ [SET_FUNCTION_ATTRIBUTE] = { .nuops = 1, .uops = { { SET_FUNCTION_ATTRIBUTE, 0, 0 } } },+ [BUILD_SLICE] = { .nuops = 1, .uops = { { BUILD_SLICE, 0, 0 } } },+ [CONVERT_VALUE] = { .nuops = 1, .uops = { { CONVERT_VALUE, 0, 0 } } },+ [FORMAT_SIMPLE] = { .nuops = 1, .uops = { { FORMAT_SIMPLE, 0, 0 } } },+ [FORMAT_WITH_SPEC] = { .nuops = 1, .uops = { { FORMAT_WITH_SPEC, 0, 0 } } },+ [COPY] = { .nuops = 1, .uops = { { COPY, 0, 0 } } },+ [SWAP] = { .nuops = 1, .uops = { { SWAP, 0, 0 } } },+}; #endifdiff --git a/Python/optimizer.c b/Python/optimizer.cindex 95cd7824e32e4..0a6cc5c92f04f 100644--- a/Python/optimizer.c+++ b/Python/optimizer.c@@ -3,7 +3,9 @@ #include "opcode.h" #include "pycore_interp.h" #include "pycore_opcode.h"+#include "opcode_metadata.h" #include "pycore_pystate.h"+#include "pycore_uops.h" #include "cpython/optimizer.h" #include <stdbool.h> #include <stdint.h>@@ -278,3 +280,200 @@ PyUnstable_Optimizer_NewCounter(void) opt->count = 0; return (PyObject *)opt; }++///////////////////// Experimental UOp Optimizer /////////////////////++#ifdef Py_DEBUG+ /* For debugging the interpreter: */+# define LLTRACE 1 /* Low-level trace feature */+#endif++static void+uop_dealloc(_PyUOpExecutorObject *self) {+ PyObject_Free(self);+}++static PyTypeObject UOpExecutor_Type = {+ PyVarObject_HEAD_INIT(&PyType_Type, 0)+ .tp_name = "uop_executor",+ .tp_basicsize = sizeof(_PyUOpExecutorObject),+ .tp_itemsize = 0,+ .tp_flags = Py_TPFLAGS_DEFAULT | Py_TPFLAGS_DISALLOW_INSTANTIATION,+ .tp_dealloc = (destructor)uop_dealloc,+};++static int+translate_bytecode_to_trace(+ PyCodeObject *code,+ _Py_CODEUNIT *instr,+ _PyUOpInstruction *trace,+ int max_length)+{+#ifdef LLTRACE+ char *uop_debug = Py_GETENV("PYTHONUOPSDEBUG");+ int lltrace = 0;+ if (uop_debug != NULL && *uop_debug >= '0') {+ lltrace = *uop_debug - '0'; // TODO: Parse an int and all that+ }+ if (lltrace >= 4) {+ fprintf(stderr,+ "Optimizing %s (%s:%d) at offset %ld\n",+ PyUnicode_AsUTF8(code->co_qualname),+ PyUnicode_AsUTF8(code->co_filename),+ code->co_firstlineno,+ (long)(instr - (_Py_CODEUNIT *)code->co_code_adaptive));+ }+#define ADD_TO_TRACE(OPCODE, OPERAND) \+ if (lltrace >= 2) { \+ const char *opname = (OPCODE) < 256 ? _PyOpcode_OpName[(OPCODE)] : ""; \+ fprintf(stderr, " ADD_TO_TRACE(%s %d, %" PRIu64 ")\n", opname, (OPCODE), (uint64_t)(OPERAND)); \+ } \+ trace[trace_length].opcode = (OPCODE); \+ trace[trace_length].operand = (OPERAND); \+ trace_length++;+#else+#define ADD_TO_TRACE(OPCODE, OPERAND) \+ trace[trace_length].opcode = (OPCODE); \+ trace[trace_length].operand = (OPERAND); \+ trace_length++;+#endif++ int trace_length = 0;+ // Always reserve space for one uop, plus SET_UP, plus EXIT_TRACE+ while (trace_length + 3 <= max_length) {+ int opcode = instr->op.code;+ uint64_t operand = instr->op.arg;+ switch (opcode) {+ case LOAD_FAST_LOAD_FAST:+ {+ // Reserve space for two uops (+ SETUP + EXIT_TRACE)+ if (trace_length + 4 > max_length) {+ goto done;+ }+ uint64_t oparg1 = operand >> 4;+ uint64_t oparg2 = operand & 15;+ ADD_TO_TRACE(LOAD_FAST, oparg1);+ ADD_TO_TRACE(LOAD_FAST, oparg2);+ break;+ }+ default:+ {+ const struct opcode_macro_expansion *expansion = &_PyOpcode_macro_expansion[opcode];+ if (expansion->nuops > 0) {+ // Reserve space for nuops (+ SETUP + EXIT_TRACE)+ int nuops = expansion->nuops;+ if (trace_length + nuops + 2 > max_length) {+ goto done;+ }+ for (int i = 0; i < nuops; i++) {+ int offset = expansion->uops[i].offset;+ switch (expansion->uops[i].size) {+ case 0:+ break;+ case 1:+ operand = read_u16(&instr[offset].cache);+ break;+ case 2:+ operand = read_u32(&instr[offset].cache);+ break;+ case 4:+ operand = read_u64(&instr[offset].cache);+ break;+ default:+ fprintf(stderr,+ "opcode=%d, operand=%" PRIu64 "; nuops=%d, i=%d; size=%d, offset=%d\n",+ opcode, operand, nuops, i,+ expansion->uops[i].size,+ expansion->uops[i].offset);+ Py_FatalError("garbled expansion");+ }+ ADD_TO_TRACE(expansion->uops[i].uop, operand);+ assert(expansion->uops[0].size == 0); // TODO+ }+ break;+ }+ // fprintf(stderr, "Unsupported opcode %d\n", opcode);+ goto done; // Break out of while loop+ }+ }+ instr++;+ // Add cache size for opcode+ instr += _PyOpcode_Caches[_PyOpcode_Deopt[opcode]];+ ADD_TO_TRACE(SET_IP, (int)(instr - (_Py_CODEUNIT *)code->co_code_adaptive));+ }+done:+ if (trace_length > 0) {+ ADD_TO_TRACE(EXIT_TRACE, 0);+#ifdef LLTRACE+ if (lltrace >= 1) {+ fprintf(stderr,+ "Created a trace for %s (%s:%d) at offset %ld -- length %d\n",+ PyUnicode_AsUTF8(code->co_qualname),+ PyUnicode_AsUTF8(code->co_filename),+ code->co_firstlineno,+ (long)(instr - (_Py_CODEUNIT *)code->co_code_adaptive),+ trace_length);+ }+#endif+ }+ else {+#ifdef LLTRACE+ if (lltrace >= 4) {+ fprintf(stderr,+ "No trace for %s (%s:%d) at offset %ld\n",+ PyUnicode_AsUTF8(code->co_qualname),+ PyUnicode_AsUTF8(code->co_filename),+ code->co_firstlineno,+ (long)(instr - (_Py_CODEUNIT *)code->co_code_adaptive));+ }+#endif+ }+ return trace_length;++#undef ADD_TO_TRACE+}++static int+uop_optimize(+ _PyOptimizerObject *self,+ PyCodeObject *code,+ _Py_CODEUNIT *instr,+ _PyExecutorObject **exec_ptr)+{+ _PyUOpInstruction trace[_Py_UOP_MAX_TRACE_LENGTH];+ int trace_length = translate_bytecode_to_trace(code, instr, trace, _Py_UOP_MAX_TRACE_LENGTH);+ if (trace_length <= 0) {+ // Error or nothing translated+ return trace_length;+ }+ OBJECT_STAT_INC(optimization_traces_created);+ _PyUOpExecutorObject *executor = (_PyUOpExecutorObject *)_PyObject_New(&UOpExecutor_Type);+ if (executor == NULL) {+ return -1;+ }+ executor->base.execute = _PyUopExecute;+ memcpy(executor->trace, trace, trace_length * sizeof(_PyUOpInstruction));+ *exec_ptr = (_PyExecutorObject *)executor;+ return 1;+}++static PyTypeObject UOpOptimizer_Type = {+ PyVarObject_HEAD_INIT(&PyType_Type, 0)+ .tp_name = "uop_optimizer",+ .tp_basicsize = sizeof(_PyOptimizerObject),+ .tp_itemsize = 0,+ .tp_flags = Py_TPFLAGS_DEFAULT | Py_TPFLAGS_DISALLOW_INSTANTIATION,+};++PyObject *+PyUnstable_Optimizer_NewUOpOptimizer(void)+{+ _PyOptimizerObject *opt = (_PyOptimizerObject *)_PyObject_New(&UOpOptimizer_Type);+ if (opt == NULL) {+ return NULL;+ }+ opt->optimize = uop_optimize;+ opt->resume_threshold = UINT16_MAX;+ opt->backedge_threshold = 0;+ return (PyObject *)opt;+}diff --git a/Python/pylifecycle.c b/Python/pylifecycle.cindex 5a5b14fbb0314..6117f3a19a550 100644--- a/Python/pylifecycle.c+++ b/Python/pylifecycle.c@@ -1181,6 +1181,19 @@ init_interp_main(PyThreadState *tstate) #endif } + // Turn on experimental tier 2 (uops-based) optimizer+ if (is_main_interp) {+ char *envvar = Py_GETENV("PYTHONUOPS");+ int enabled = envvar != NULL && *envvar > '0';+ if (_Py_get_xoption(&config->xoptions, L"uops") != NULL) {+ enabled = 1;+ }+ if (enabled) {+ PyObject *opt = PyUnstable_Optimizer_NewUOpOptimizer();+ PyUnstable_SetOptimizer((_PyOptimizerObject *)opt);+ }+ }+ assert(!_PyErr_Occurred(tstate)); return _PyStatus_OK();diff --git a/Python/specialize.c b/Python/specialize.cindex 0006aa733bd6c..3f51432a63af1 100644--- a/Python/specialize.c+++ b/Python/specialize.c@@ -195,6 +195,10 @@ print_object_stats(FILE *out, ObjectStats *stats) fprintf(out, "Object method cache collisions: %" PRIu64 "\n", stats->type_cache_collisions); fprintf(out, "Object method cache dunder hits: %" PRIu64 "\n", stats->type_cache_dunder_hits); fprintf(out, "Object method cache dunder misses: %" PRIu64 "\n", stats->type_cache_dunder_misses);+ fprintf(out, "Optimization attempts: %" PRIu64 "\n", stats->optimization_attempts);+ fprintf(out, "Optimization traces created: %" PRIu64 "\n", stats->optimization_traces_created);+ fprintf(out, "Optimization traces executed: %" PRIu64 "\n", stats->optimization_traces_executed);+ fprintf(out, "Optimization uops executed: %" PRIu64 "\n", stats->optimization_uops_executed); } static voiddiff --git a/Tools/c-analyzer/cpython/_parser.py b/Tools/c-analyzer/cpython/_parser.pyindex 51b79c25f9e20..9bc7285e18b2f 100644--- a/Tools/c-analyzer/cpython/_parser.py+++ b/Tools/c-analyzer/cpython/_parser.py@@ -83,6 +83,7 @@ def clean_lines(text): Python/deepfreeze/*.c Python/frozen_modules/*.h Python/generated_cases.c.h+Python/executor_cases.c.h # not actually source Python/bytecodes.cdiff --git a/Tools/c-analyzer/cpython/ignored.tsv b/Tools/c-analyzer/cpython/ignored.tsvindex 9cce3f3b25c18..73eec6631d951 100644--- a/Tools/c-analyzer/cpython/ignored.tsv+++ b/Tools/c-analyzer/cpython/ignored.tsv@@ -366,6 +366,8 @@ Python/sysmodule.c-whatstrings- Python/optimizer.c-DefaultOptimizer_Type- Python/optimizer.c-CounterExecutor_Type- Python/optimizer.c-CounterOptimizer_Type-+Python/optimizer.c-UOpExecutor_Type-+Python/optimizer.c-UOpOptimizer_Type- Python/optimizer.c-_PyOptimizer_Default- ##-----------------------diff --git a/Tools/cases_generator/README.md b/Tools/cases_generator/README.mdindex c595a932ac447..fc9331656fe78 100644--- a/Tools/cases_generator/README.md+++ b/Tools/cases_generator/README.md@@ -9,7 +9,7 @@ What's currently here: - `plexer.py`: OO interface on top of lexer.py; main class: `PLexer` - `parser.py`: Parser for instruction definition DSL; main class `Parser` - `generate_cases.py`: driver script to read `Python/bytecodes.c` and- write `Python/generated_cases.c.h`+ write `Python/generated_cases.c.h` (and several other files) - `test_generator.py`: tests, require manual running using `pytest` Note that there is some dummy C code at the top and bottom ofdiff --git a/Tools/cases_generator/generate_cases.py b/Tools/cases_generator/generate_cases.pyindex 1afdeef41f0ef..afeb31dd441ae 100644--- a/Tools/cases_generator/generate_cases.py+++ b/Tools/cases_generator/generate_cases.py@@ -29,6 +29,9 @@ DEFAULT_PYMETADATA_OUTPUT = os.path.relpath( os.path.join(ROOT, "Lib/_opcode_metadata.py") )+DEFAULT_EXECUTOR_OUTPUT = os.path.relpath(+ os.path.join(ROOT, "Python/executor_cases.c.h")+) BEGIN_MARKER = "// BEGIN BYTECODES //" END_MARKER = "// END BYTECODES //" RE_PREDICTED = (@@ -61,6 +64,13 @@ arg_parser.add_argument( "input", nargs=argparse.REMAINDER, help="Instruction definition file(s)" )+arg_parser.add_argument(+ "-e",+ "--executor-cases",+ type=str,+ help="Write executor cases to this file",+ default=DEFAULT_EXECUTOR_OUTPUT,+) def effect_size(effect: StackEffect) -> tuple[int, str]:@@ -176,14 +186,14 @@ def indent(self): self.prefix = self.prefix[:-4] @contextlib.contextmanager- def block(self, head: str):+ def block(self, head: str, tail: str = ""): if head: self.emit(head + " {") else: self.emit("{") with self.indent(): yield- self.emit("}")+ self.emit("}" + tail) def stack_adjust( self,@@ -290,6 +300,29 @@ def emit_macros(cls, out: Formatter): f"(_PyOpcode_opcode_metadata[(OP)].flags & ({name}))") +FORBIDDEN_NAMES_IN_UOPS = (+ "resume_with_error", # Proxy for "goto", which isn't an IDENTIFIER+ "unbound_local_error",+ "kwnames",+ "next_instr",+ "oparg1", # Proxy for super-instructions like LOAD_FAST_LOAD_FAST+ "JUMPBY",+ "DISPATCH",+ "INSTRUMENTED_JUMP",+ "throwflag",+ "exception_unwind",+ "import_from",+ "import_name",+ "_PyObject_CallNoArgs", # Proxy for BEFORE_WITH+)+++# Interpreter tiers+TIER_ONE = 1 # Specializing adaptive interpreter (PEP 659)+TIER_TWO = 2 # Experimental tracing interpreter+Tiers: typing.TypeAlias = typing.Literal[1, 2]++ @dataclasses.dataclass class Instruction: """An instruction with additional data and code."""@@ -353,7 +386,32 @@ def __init__(self, inst: parser.InstDef): cache = "0" self.instr_fmt = fmt - def write(self, out: Formatter) -> None:+ def is_viable_uop(self) -> bool:+ """Whether this instruction is viable as a uop."""+ if self.always_exits:+ return False+ if self.instr_flags.HAS_ARG_FLAG:+ # If the instruction uses oparg, it cannot use any caches+ for c in self.cache_effects:+ if c.name != UNUSED:+ return False+ else:+ # If it doesn't use oparg, it can have one cache entry+ caches: list[parser.CacheEffect] = []+ cache_offset = 0+ for c in self.cache_effects:+ if c.name != UNUSED:+ caches.append(c)+ cache_offset += c.size+ if len(caches) > 1:+ return False+ for forbidden in FORBIDDEN_NAMES_IN_UOPS:+ # TODO: Don't check in '#ifdef ENABLE_SPECIALIZATION' regions+ if variable_used(self.inst, forbidden):+ return False+ return True++ def write(self, out: Formatter, tier: Tiers = TIER_ONE) -> None: """Write one instruction, sans prologue and epilogue.""" # Write a static assertion that a family's cache size is correct if family := self.family:@@ -400,7 +458,7 @@ def write(self, out: Formatter) -> None: # out.emit(f"next_instr += OPSIZE({self.inst.name}) - 1;") - self.write_body(out, 0)+ self.write_body(out, 0, tier=tier) # Skip the rest if the block always exits if self.always_exits:@@ -427,10 +485,16 @@ def write(self, out: Formatter) -> None: out.assign(dst, oeffect) # Write cache effect- if self.cache_offset:+ if tier == TIER_ONE and self.cache_offset: out.emit(f"next_instr += {self.cache_offset};") - def write_body(self, out: Formatter, dedent: int, cache_adjust: int = 0) -> None:+ def write_body(+ self,+ out: Formatter,+ dedent: int,+ cache_adjust: int = 0,+ tier: Tiers = TIER_ONE,+ ) -> None: """Write the instruction body.""" # Write cache effect variable declarations and initializations cache_offset = cache_adjust@@ -447,9 +511,12 @@ def write_body(self, out: Formatter, dedent: int, cache_adjust: int = 0) -> None else: typ = f"uint{bits}_t " func = f"read_u{bits}"- out.emit(- f"{typ}{ceffect.name} = {func}(&next_instr[{cache_offset}].cache);"- )+ if tier == TIER_ONE:+ out.emit(+ f"{typ}{ceffect.name} = {func}(&next_instr[{cache_offset}].cache);"+ )+ else:+ out.emit(f"{typ}{ceffect.name} = operand;") cache_offset += ceffect.size assert cache_offset == self.cache_offset + cache_adjust @@ -573,16 +640,24 @@ class Analyzer: output_filename: str metadata_filename: str pymetadata_filename: str+ executor_filename: str errors: int = 0 emit_line_directives: bool = False - def __init__(self, input_filenames: list[str], output_filename: str,- metadata_filename: str, pymetadata_filename: str):+ def __init__(+ self,+ input_filenames: list[str],+ output_filename: str,+ metadata_filename: str,+ pymetadata_filename: str,+ executor_filename: str,+ ): """Read the input file.""" self.input_filenames = input_filenames self.output_filename = output_filename self.metadata_filename = metadata_filename self.pymetadata_filename = pymetadata_filename+ self.executor_filename = executor_filename def error(self, msg: str, node: parser.Node) -> None: lineno = 0@@ -1107,6 +1182,8 @@ def write_metadata(self) -> None: self.write_pseudo_instrs() + self.write_uop_defines()+ self.write_stack_effect_functions() # Write type definitions@@ -1114,12 +1191,17 @@ def write_metadata(self) -> None: InstructionFlags.emit_macros(self.out) - self.out.emit("struct opcode_metadata {")- with self.out.indent():+ with self.out.block("struct opcode_metadata", ";"): self.out.emit("bool valid_entry;") self.out.emit("enum InstructionFormat instr_format;") self.out.emit("int flags;")- self.out.emit("};")+ self.out.emit("")++ with self.out.block("struct opcode_macro_expansion", ";"):+ self.out.emit("int nuops;")+ self.out.emit("struct { int16_t uop; int8_t size; int8_t offset; } uops[8];")+ self.out.emit("")+ self.out.emit("") self.out.emit("#define OPCODE_METADATA_FMT(OP) " "(_PyOpcode_opcode_metadata[(OP)].instr_format)")@@ -1130,7 +1212,9 @@ def write_metadata(self) -> None: # Write metadata array declaration self.out.emit("#ifndef NEED_OPCODE_METADATA") self.out.emit("extern const struct opcode_metadata _PyOpcode_opcode_metadata[512];")+ self.out.emit("extern const struct opcode_macro_expansion _PyOpcode_macro_expansion[256];") self.out.emit("#else")+ self.out.emit("const struct opcode_metadata _PyOpcode_opcode_metadata[512] = {") # Write metadata for each instruction@@ -1150,6 +1234,31 @@ def write_metadata(self) -> None: # Write end of array self.out.emit("};")++ with self.out.block(+ "const struct opcode_macro_expansion _PyOpcode_macro_expansion[256] =",+ ";",+ ):+ # Write macro expansion for each non-pseudo instruction+ for thing in self.everything:+ match thing:+ case OverriddenInstructionPlaceHolder():+ pass+ case parser.InstDef(name=name):+ instr = self.instrs[name]+ if instr.kind != "op" and instr.is_viable_uop():+ self.out.emit(+ f"[{name}] = "+ f"{{ .nuops = 1, .uops = {{ {{ {name}, 0, 0 }} }} }},"+ )+ case parser.Macro():+ # TODO: emit expansion if all parts are viable uops+ pass+ case parser.Pseudo():+ pass+ case _:+ typing.assert_never(thing)+ self.out.emit("#endif") with open(self.pymetadata_filename, "w") as f:@@ -1184,7 +1293,6 @@ def write_metadata(self) -> None: "opcode for family in _specializations.values() for opcode in family" "]") - def write_pseudo_instrs(self) -> None: """Write the IS_PSEUDO_INSTR macro""" self.out.emit("\n\n#define IS_PSEUDO_INSTR(OP) \\")@@ -1192,6 +1300,20 @@ def write_pseudo_instrs(self) -> None: self.out.emit(f" ((OP) == {op}) || \\") self.out.emit(f" 0") + def write_uop_defines(self) -> None:+ """Write '#define XXX NNN' for each uop"""+ self.out.emit("")+ counter = 300+ def add(name: str) -> None:+ nonlocal counter+ self.out.emit(f"#define {name} {counter}")+ counter += 1+ add("EXIT_TRACE")+ add("SET_IP")+ for instr in self.instrs.values():+ if instr.kind == "op" and instr.is_viable_uop():+ add(instr.name)+ def emit_metadata_entry( self, name: str, fmt: str, flags: InstructionFlags ) -> None:@@ -1221,10 +1343,7 @@ def write_instructions(self) -> None: # Create formatter self.out = Formatter(f, 8, self.emit_line_directives) - # Write provenance header- self.out.write_raw(f"{self.out.comment} This file is generated by {THIS}\n")- self.out.write_raw(self.from_source_files())- self.out.write_raw(f"{self.out.comment} Do not edit!\n")+ self.write_provenance_header() # Write and count instructions of all kinds n_instrs = 0@@ -1252,6 +1371,33 @@ def write_instructions(self) -> None: file=sys.stderr, ) + def write_executor_instructions(self) -> None:+ """Generate cases for the Tier 2 interpreter."""+ with open(self.executor_filename, "w") as f:+ self.out = Formatter(f, 8)+ self.write_provenance_header()+ for thing in self.everything:+ match thing:+ case OverriddenInstructionPlaceHolder():+ self.write_overridden_instr_place_holder(thing)+ case parser.InstDef():+ instr = self.instrs[thing.name]+ if instr.is_viable_uop():+ self.out.emit("")+ with self.out.block(f"case {thing.name}:"):+ instr.write(self.out, tier=TIER_TWO)+ self.out.emit("break;")+ case parser.Macro():+ pass # TODO+ case parser.Pseudo():+ pass+ case _:+ typing.assert_never(thing)+ print(+ f"Wrote some stuff to {self.executor_filename}",+ file=sys.stderr,+ )+ def write_overridden_instr_place_holder(self, place_holder: OverriddenInstructionPlaceHolder) -> None: self.out.emit("")@@ -1405,7 +1551,7 @@ def main(): args.input.append(DEFAULT_INPUT) # Raises OSError if input unreadable- a = Analyzer(args.input, args.output, args.metadata, args.pymetadata)+ a = Analyzer(args.input, args.output, args.metadata, args.pymetadata, args.executor_cases) if args.emit_line_directives: a.emit_line_directives = True@@ -1415,6 +1561,7 @@ def main(): sys.exit(f"Found {a.errors} errors") a.write_instructions() # Raises OSError if output can't be written a.write_metadata()+ a.write_executor_instructions() if __name__ == "__main__":diff --git a/Tools/cases_generator/test_generator.py b/Tools/cases_generator/test_generator.pyindex 9ff4c971fd21a..e374ac41e6a94 100644--- a/Tools/cases_generator/test_generator.py+++ b/Tools/cases_generator/test_generator.py@@ -44,7 +44,15 @@ def run_cases_test(input: str, expected: str): temp_input.flush() temp_output = tempfile.NamedTemporaryFile("w+") temp_metadata = tempfile.NamedTemporaryFile("w+")- a = generate_cases.Analyzer([temp_input.name], temp_output.name, temp_metadata.name)+ temp_pymetadata = tempfile.NamedTemporaryFile("w+")+ temp_executor = tempfile.NamedTemporaryFile("w+")+ a = generate_cases.Analyzer(+ [temp_input.name],+ temp_output.name,+ temp_metadata.name,+ temp_pymetadata.name,+ temp_executor.name,+ ) a.parse() a.analyze() if a.errors:
Reply
Sign in to reply online Use email software
![Mailman 3
[Python-checkins] gh-104584: Baby steps towards generating and executing traces (#105924) - Python-checkins (2024)](data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mP8/h8AAvMB+NzmkbcAAAAASUVORK5CYII=)