{"id":329454,"date":"2023-05-29T16:18:00","date_gmt":"2023-05-29T16:18:00","guid":{"rendered":"http:\/\/itteacheritfreelance.hk\/wordpress\/?guid=60d61c990fc2ecdcaa05b17fff362ea9"},"modified":"2023-05-29T16:18:00","modified_gmt":"2023-05-29T16:18:00","slug":"the-python-language-summit-2023-making-the-global-interpreter-lock-optional","status":"publish","type":"post","link":"https:\/\/itteacheritfreelance.hk\/wordpress\/index.php\/2023\/05\/29\/the-python-language-summit-2023-making-the-global-interpreter-lock-optional\/","title":{"rendered":"The Python Language Summit 2023: Making the Global Interpreter Lock Optional"},"content":{"rendered":"

<\/p>\n

The Global Interpreter Lock (\u201cGIL\u201d), is one of the most fundamental parts of how Python works today. It\u2019s also one of the most controversial parts, as it prevents true concurrency between threads \u2013 another way of saying that it\u2019s difficult to run two functions simultaneously while writing pure-Python code.<\/p>\n

If there\u2019s one blog that really \u201ctook off\u201d after I wrote last year\u2019s coverage on the Python Language Summit, it was <\/span>my blog on Sam Gross\u2019s proposal to make Python\u2019s Global Interpreter Lock (the \u201cGIL\u201d) optional<\/span><\/a>. One week following the publication of my articles, the blog had been viewed nearly 38,000 times; the blog in \u201csecond place\u201d had only been viewed 5,300 times.<\/span><\/p>\n

Interest in removing the GIL is clear, therefore \u2013 and this year, Gross returned to the Python Language Summit<\/a> to discuss the development of his plans.<\/span><\/p>\n

Dare to dream of a GIL-free world<\/span><\/h2>\n

Gross started off by giving an update on how work on <\/span>nogil<\/code> \u2013 Gross\u2019s fork of CPython with the GIL removed \u2013 had progressed over the past year. Gross had been spending the last few months rebasing his fork onto CPython 3.12. As a result, he was now able to give an accurate estimate of how bad the performance costs to single-threaded code would be, if the GIL were removed.<\/span><\/p>\n

Over the past year, Gross had also written a PEP \u2013 <\/span>PEP 703<\/span><\/a> \u2013 which, following the Language Summit, was <\/span>submitted to the Steering Council for their consideration<\/span><\/a> on May 12. If the PEP is accepted by the Steering Council, a version of Python with the GIL disabled could be available as soon as Python 3.13. (To discuss the ideas in the PEP, head to <\/span>the thread on discuss.python.org<\/span><\/a>.)<\/span><\/p>\n

Gross reported that the latest version of <\/span>nogil<\/code> was around 6% slower on single-threaded code than the CPython <\/span>main<\/code> branch, and that he was confident that the performance overhead could be reduced even further, possibly to nearly 0%. Most of the overhead was due to reference counting and changes that had been required to the operation of CPython\u2019s new-in-3.11 <\/span>specialising adaptive interpreter<\/span><\/a>. With multiple threads, the performance overhead was around 8%.<\/span><\/p>\n


<\/span><\/p>\n<\/p>\n\n\n\n\n
<\/a><\/td>\n<\/tr>\n
A slide from Gross’s talk
(numbers indicate the per-thread overhead)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

<\/p>\n

Having rebased onto Python 3.12, Gross was also able to give an estimate on the extent of the code changes that would be required to CPython, were <\/span>nogil<\/code> to be accepted. Excluding generated files, Gross reported that around 15,000 lines of code would need to be changed. There would also need to be some alterations to <\/span>mimalloc<\/span><\/a>, which <\/span>nogil<\/code> depends on \u2013 but Gross was hopeful that these would be accepted to mimalloc itself, reducing the need for CPython to maintain mimalloc patches in its codebase.<\/span><\/p>\n

Some audience members expressed concern about the prospect of \u201c<\/span>#ifdef<\/code> hell\u201d across the code base, to account for the two possible CPython build options. However, Gross countered that most of the changes required could be made unconditionally without resorting to <\/span>#ifdef<\/code>s, since the changes were simply no-ops with the GIL enabled.<\/span><\/p>\n

The plan for <\/span>nogil<\/code> remains that it would be enabled via a compile-time flag, named <\/span>--disable-gil<\/code>. Third-party C extensions would need to provide separate wheels for GIL-disabled Python.<\/span><\/p>\n

GIL gotta go<\/span><\/h2>\n

Last year, it felt as though Gross\u2019s proposal was met with a certain degree of excitement, but also a certain degree of scepticism. It was thrilling to see how far <\/span>nogil<\/code> had come, but there was a certain amount of frustration in the room at the lack of a concrete plan for where to go next.<\/span><\/p>\n

This year, it felt like the proposal was greeted much more warmly. Gross had come to the summit with a far more clearly defined roadmap for how we might get to <\/span>nogil<\/code>, and had taken the time to put together detailed estimates of the performance impact and the extent of the code changes.<\/span><\/p>\n

\n

\u201cThank you for doing this\u2026 now we have something we can actually consider!\u201d<\/span> <\/p>\n<\/blockquote>\n

\n

— Brandt Bucher, CPython Core Developer<\/i><\/p>\n<\/blockquote>\n

Attendees at the Language Summit were also impressed at how low the performance overhead of <\/span>nogil<\/code> was, although Mark Shannon commented that he thought the numbers might be \u201ca slight underestimate\u201d. Gross explained that he had managed to achieve a 2-3% speedup by optimising for the case where only a single thread was active. Even in multithreaded programs, he explained, this provided a performance boost, since even in multithreaded code, it was often the case that only a single thread would be attempting to access an object at any given point in time.<\/span><\/p>\n

Larry Hastings expressed concern that <\/span>nogil<\/code> might make debugging code harder; Gross responded that there was some impact on debuggability, but that it wasn\u2019t too bad, in his opinion. Pablo Galindo Salgado, release manager for Python 3.10 and 3.11, expressed concern that <\/span>nogil<\/code> could also make implementing debuggers trickier \u2013 but commented that \u201cit might be worth the price\u201d anyway.<\/span><\/p>\n

Another point of discussion was the changes Gross had made to <\/span>the specialising adaptive interpreter<\/span><\/a> in the <\/span>nogil<\/code> fork. In order for the specialisations to work with the GIL disabled, Gross had had to guard the adaptive specialisations to the bytecode behind a lock. As well as this, each thread had been limited to a single specialisation of any given bytecode; with the GIL enabled, the adaptive interpreter can respecialise bytecode multiple times. Gross commented that he thought it would probably be possible to allow multiple specialisations of bytecode in multithreaded code, but that this would require further investigation. His current solution was the simplest one he had found, for now.<\/span><\/p>\n<\/div>\n\n

\\\"IT\u96fb\u8166\u88dc\u7fd2
\u7acb\u523b\u8a3b\u518a\u53ca\u5831\u540d\u96fb\u8166\u88dc\u7fd2\u8ab2\u7a0b\u5427!<\/figcaption><\/figure>\r\n<\/br>Find A Teacher Form:\r\n<\/br>https:\/\/docs.google.com\/forms\/d\/1vREBnX5n262umf4wU5U2pyTwvk9O-JrAgblA-wH9GFQ\/viewform?edit_requested=true#responses\r\n<\/br><\/br>Email:\r\n<\/br>public1989two@gmail.com






\r\nwww.itsec.hk<\/a>
\r\nwww.itsec.vip<\/a>
\r\nwww.itseceu.uk<\/a>
<\/p>","protected":false},"excerpt":{"rendered":"

The Global Interpreter Lock (\u201cGIL\u201d), is one of the most fundamental parts of how Python works today. It\u2019s also one of the most controversial parts, as it prevents true concurrency between threads \u2013 another way of saying that it\u2019s difficult to run two …<\/p>\n<\/div>","protected":false},"author":2053,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"slim_seo":{"title":"The Python Language Summit 2023: Making the Global Interpreter Lock Optional - ITTeacherITFreelance.hk","description":"The Global Interpreter Lock (\u201cGIL\u201d), is one of the most fundamental parts of how Python works today. It\u2019s also one of the most controversial parts, as it preven"},"footnotes":""},"categories":[10700],"tags":[],"_links":{"self":[{"href":"https:\/\/itteacheritfreelance.hk\/wordpress\/index.php\/wp-json\/wp\/v2\/posts\/329454"}],"collection":[{"href":"https:\/\/itteacheritfreelance.hk\/wordpress\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/itteacheritfreelance.hk\/wordpress\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/itteacheritfreelance.hk\/wordpress\/index.php\/wp-json\/wp\/v2\/users\/2053"}],"replies":[{"embeddable":true,"href":"https:\/\/itteacheritfreelance.hk\/wordpress\/index.php\/wp-json\/wp\/v2\/comments?post=329454"}],"version-history":[{"count":1,"href":"https:\/\/itteacheritfreelance.hk\/wordpress\/index.php\/wp-json\/wp\/v2\/posts\/329454\/revisions"}],"predecessor-version":[{"id":329455,"href":"https:\/\/itteacheritfreelance.hk\/wordpress\/index.php\/wp-json\/wp\/v2\/posts\/329454\/revisions\/329455"}],"wp:attachment":[{"href":"https:\/\/itteacheritfreelance.hk\/wordpress\/index.php\/wp-json\/wp\/v2\/media?parent=329454"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/itteacheritfreelance.hk\/wordpress\/index.php\/wp-json\/wp\/v2\/categories?post=329454"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/itteacheritfreelance.hk\/wordpress\/index.php\/wp-json\/wp\/v2\/tags?post=329454"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}