Skip to content

Refactor codecs error handlers to use _PyUnicodeError_GetParams and extract complex logic into separate functions #129173

Closed
Closed
Refactor codecs error handlers to use _PyUnicodeError_GetParams and extract complex logic into separate functions#129173
Assignees
picnixz
Labels
interpreter-core(Objects, Python, Grammar, and Parser dirs)topic-unicodetype-featureA feature request or enhancement

Description

@picnixz
picnixz
opened on Jan 22, 2025

Feature or enhancement

Proposal:

I want to refactor the different codecs handlers in Python/codecs.c to use _PyUnicodeError_GetParams. Some codecs handlers will be refactored as part of #126004 but some others are not subject to issues (namely, the ignore, namereplace, surrogateescape, and surrogatepass handlers do not suffer from crashes, or at least I wasn't able to make them crash easily).

In addition, I also plan to split the handlers into functions instead of 2 or 3 big blocks of code handling a specific exception. For that reason, I will introduce the following helper macros:

#define _PyIsUnicodeEncodeError(EXC)    \
    PyObject_TypeCheck(EXC, (PyTypeObject *)PyExc_UnicodeEncodeError)
#define _PyIsUnicodeDecodeError(EXC)    \
    PyObject_TypeCheck(EXC, (PyTypeObject *)PyExc_UnicodeDecodeError)
#define _PyIsUnicodeTranslateError(EXC) \
    PyObject_TypeCheck(EXC, (PyTypeObject *)PyExc_UnicodeTranslateError)

For handlers that need to be fixed, I will first fix them in-place (no refactorization). Afterwards, I will refactor them and extract the relevant part of the code into functions. That way, the diff will be easier to follow (I've observed that it's much harder to read the diff where I did both so I will revert that part in the existing PRs; EDIT: actually there is no PR doing both fixes and split...).

I'm creating this issue to track the progression of the refactorization if no issue occurs.

cc @vstinner @encukou

Has this already been discussed elsewhere?

This is a minor feature, which does not need previous discussion elsewhere

Links to previous discussion of this feature:

No response

Linked PRs

Activity

  1. picnixz
    added
    interpreter-core(Objects, Python, Grammar, and Parser dirs)
    type-featureA feature request or enhancement
    on Jan 22, 2025
  2. picnixz
    self-assigned this
    on Jan 22, 2025

  3. picnixz commented on Jan 22, 2025

    @picnixz
    picnixz
    on Jan 22, 2025
    MemberAuthor

    The merge plan is as follows:

  4. picnixz
    added this to Codecs and encodings issueson Jan 23, 2025
  5. picnixz
    added a commit that references this issue on Jan 24, 2025

    gh-129173: Use `_PyUnicodeError_GetParams` in `PyCodec_IgnoreErrors` (#…

    36bb229
  6. encukou
    added a commit that references this issue on Feb 8, 2025

    gh-129173: Use `_PyUnicodeError_GetParams` in `PyCodec_NameReplaceErr…

    a56ead0
  7. picnixz
    changed the title [-]Refactor codecs error handlers to use `_PyUnicodeError_GetParams`[/-] [+]Refactor codecs error handlers to use `_PyUnicodeError_GetParams` and extract logic into separate functions[/+] on Feb 9, 2025
  8. picnixz
    changed the title [-]Refactor codecs error handlers to use `_PyUnicodeError_GetParams` and extract logic into separate functions[/-] [+]Refactor codecs error handlers to use `_PyUnicodeError_GetParams` and extract complex logic into separate functions[/+] on Feb 9, 2025
  9. encukou
    added 2 commits that reference this issue on Feb 14, 2025

    gh-129173: Use `_PyUnicodeError_GetParams` in `PyCodec_SurrogatePassE…

    1775091

    gh-129173: Use `_PyUnicodeError_GetParams` in `PyCodec_SurrogateEscap…

    e24a1ac
  10. picnixz
    added 2 commits that reference this issue on Feb 25, 2025

    gh-129173: refactor `PyCodec_ReplaceErrors` into separate functions (#…

    fa6a814

    gh-129173: simplify `PyCodec_XMLCharRefReplaceErrors` logic (#129894)

    f693f84
  11. picnixz
    added
    topic-unicode
    on Mar 3, 2025
  12. picnixz
    added a commit that references this issue on Mar 3, 2025

    gh-129173: refactor `PyCodec_BackslashReplaceErrors` into separate fu…

    3146a25

  13. picnixz commented on Mar 3, 2025

    @picnixz
    picnixz
    on Mar 3, 2025
    MemberAuthor

    We're done with this task! next one is to address #129894 (comment).

  14. picnixz
    closed this as completedon Mar 4, 2025
  15. github-project-automation
    moved this to Done in Codecs and encodings issueson Mar 4, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

interpreter-core(Objects, Python, Grammar, and Parser dirs)topic-unicodetype-featureA feature request or enhancement

Projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions