Skip to content

[BUG] Setuptools via pyproject.toml parses maintainers list badly #3663

New issue
New issue
Closed
Closed
[BUG] Setuptools via pyproject.toml parses maintainers list badly#3663
Labels
Needs TriageIssues that need to be evaluated for severity and status.bug

Description

@nabriis
nabriis
opened on Nov 4, 2022

setuptools version

65.5.1

Python version

OS

Additional environment information

setuptools via isolated environment as part of pyproject.toml.

Description

Our releases build via setuptools using a pyproject.toml file have just now started to fail because the maintainer list added to METADATA is some odd string rather than the list of maintainers.

It has been working at least a few days ago.

The pyproject.toml file is here https://github.com/CUQI-DTU/CUQIpy/blob/main/pyproject.toml. Relevant parts are:

[build-system]
requires = ["setuptools>=42", "wheel", "versioneer[toml]==0.26"]
build-backend = "setuptools.build_meta"

[project]
maintainers = [
    {name = "Nicolai A. B. Riis", email = "nabr@dtu.dk"},
    {name = "Jakob S. Jørgensen", email = "jakj@dtu.dk"},
    {name = "Amal M. Alghamdi", email = "amaal@dtu.dk"}
]

Somehow in the METADATA after running python -m build --sdist --wheel --outdir dist/ . maintainers is written as follows.

'=?utf-8?b?Ik5pY29sYWkgQS4gQi4g...bWRpIiA8YW1hYWxAZHR1LmRrPg==?='

Error thrown by PyPI when attempting to upload release can be viewed here:
https://github.com/CUQI-DTU/CUQIpy/actions/runs/3397562988/jobs/5649833961

Perhaps related to the ø in one of our names?

Expected behavior

Expect to correctly parse the maintainer list and provide METADATA file with correct format of maintainers.

How to Reproduce

  1. Create Python project with pyproject.toml file and maintainer list as written above.
  2. Build wheels etc. for release
  3. Check METADATA file

Output

Activity
  1. nabriis
    added 
    bug
     
    Needs TriageIssues that need to be evaluated for severity and status.
     on Nov 4, 2022
  2. nabriis
    changed the title [-][BUG] Setuptools parses maintainer badly[/-] [+][BUG] Setuptools parses maintainers list badly[/+] on Nov 4, 2022
  3. nabriis
    changed the title [-][BUG] Setuptools parses maintainers list badly[/-] [+][BUG] Setuptools via pyproject.toml parses maintainers list badly[/+] on Nov 4, 2022
  4. abravalheri commented on Nov 9, 2022 
    @abravalheri
    abravalheri
    on Nov 9, 2022
    Contributor
    Hi @nabriis thank you very much for reporting this.
    

    I checked your reproducer and it does seem that setuptools is writing the PKG-INFO file in the sdist. That is curious.
    

    Motivated by this result, I did some investigation in #3666, and I believe that setuptools is (surprisingly 😅) doing the right thing here.
    

    Since setuptools uses another package pypa/wheel to create the METADATA file, I started to suspect that there might be a problem with pypa/wheel. To verify that, I did the following experiment:
    

    rm -rf /tmp/helloworld
mkdir /tmp/helloworld
cd /tmp/helloworld
cat <<EOF > pyproject.toml
[build-system]
requires = ["setuptools>=42", "wheel"]
build-backend = "setuptools.build_meta"

[project]
name = "helloworld"
version = "42"
maintainers = [
    {name = "Nicolai A. B. Riis", email = "nabr@dtu.dk"},
    {name = "Jakob S. Jørgensen", email = "jakj@dtu.dk"},
    {name = "Amal M. Alghamdi", email = "amaal@dtu.dk"}
]
EOF
virtualenv -p python3.10 .venv
.venv/bin/python -m pip install -U pip setuptools wheel build

cat <<EOF > investigation.patch
--- .venv/lib/python3.8/site-packages/wheel/bdist_wheel.py      2022-11-09 12:07:41.250225200 +0000
+++ /tmp/modified_bdist_wheel.py        2022-11-09 12:07:29.936182800 +0000
@@ -508,10 +508,16 @@
             # .egg-info is a single file
             pkginfo_path = egginfo_path
             pkg_info = pkginfo_to_metadata(egginfo_path, egginfo_path)
+            with open(pkginfo_path, "r", encoding="utf-8") as inp:
+                print("\n\n\n~-------------------------~\nBefore wheel conversion:\n")
+                print(inp.read())
             os.mkdir(distinfo_path)
         else:
             # .egg-info is a directory
             pkginfo_path = os.path.join(egginfo_path, "PKG-INFO")
+            with open(pkginfo_path, "r", encoding="utf-8") as inp:
+                print("\n\n\n~-------------------------~\nBefore wheel conversion:\n")
+                print(inp.read())
             pkg_info = pkginfo_to_metadata(egginfo_path, pkginfo_path)

             # ignore common egg metadata that is useless to wheel
@@ -537,6 +543,10 @@
         with open(pkg_info_path, "w", encoding="utf-8") as out:
             Generator(out, mangle_from_=False, maxheaderlen=0).flatten(pkg_info)

+        with open(pkg_info_path, "r", encoding="utf-8") as out:
+            print("\n\n\n~-------------------------~\nAfter wheel conversion:\n")
+            print(out.read())
+
         for license_path in self.license_paths:
             filename = os.path.basename(license_path)
             shutil.copy(license_path, os.path.join(distinfo_path, filename))
EOF

patch -u .venv/lib/python3.10/site-packages/wheel/bdist_wheel.py -i investigation.patch
.venv/bin/pyproject-build --no-isolation
    

    The following is part of the output of the experiment:
    

    # ...
Copying helloworld.egg-info to build/bdist.linux-x86_64/wheel/helloworld-42-py3.10.egg-info
running install_scripts



~-------------------------~
Before wheel conversion:

Metadata-Version: 2.1
Name: helloworld
Version: 42
Maintainer-email: "Nicolai A. B. Riis" <nabr@dtu.dk>, "Jakob S. Jørgensen" <jakj@dtu.dk>, "Amal M. Alghamdi" <amaal@dtu.dk>




~-------------------------~
After wheel conversion:

Metadata-Version: 2.1
Name: helloworld
Version: 42
Maintainer-email: =?utf-8?b?Ik5pY29sYWkgQS4gQi4gUmlpcyIgPG5hYnJAZHR1LmRrPiwgIkpha29iIFMuIErDuHJnZW5zZW4iIDxqYWtqQGR0dS5kaz4sICJBbWFsIE0uIEFsZ2hhbWRpIiA8YW1hYWxAZHR1LmRrPg==?=


creating build/bdist.linux-x86_64/wheel/helloworld-42.dist-info/WHEEL
# 
    

    The output does seems to indicate that the change in the encoding is indeed happening inside of pypa/wheel. So I think the next step is to report it to that repository (I plan to do it soon, but please feel free to beat me to that).
  5. nabriis commented on Nov 9, 2022 
    @nabriis
    nabriis
    on Nov 9, 2022
    Author
    

    Hi @nabriis thank you very much for reporting this.
    

    I checked your reproducer and it does seem that setuptools is writing the PKG-INFO file in the sdist. That is curious.
    

    Motivated by this result, I did some investigation in #3666, and I believe that setuptools is (surprisingly 😅) doing the right thing here.
    

    Since setuptools uses another package pypa/wheel to create the METADATA file, I started to suspect that there might be a problem with pypa/wheel. To verify that, I did the following experiment:
    

    rm -rf /tmp/helloworld
mkdir /tmp/helloworld
cd /tmp/helloworld
cat <<EOF > pyproject.toml
[build-system]
requires = ["setuptools>=42", "wheel"]
build-backend = "setuptools.build_meta"

[project]
name = "helloworld"
version = "42"
maintainers = [
    {name = "Nicolai A. B. Riis", email = "nabr@dtu.dk"},
    {name = "Jakob S. Jørgensen", email = "jakj@dtu.dk"},
    {name = "Amal M. Alghamdi", email = "amaal@dtu.dk"}
]
EOF
virtualenv -p python3.10 .venv
.venv/bin/python -m pip install -U pip setuptools wheel build

cat <<EOF > investigation.patch
--- .venv/lib/python3.8/site-packages/wheel/bdist_wheel.py      2022-11-09 12:07:41.250225200 +0000
+++ /tmp/modified_bdist_wheel.py        2022-11-09 12:07:29.936182800 +0000
@@ -508,10 +508,16 @@
             # .egg-info is a single file
             pkginfo_path = egginfo_path
             pkg_info = pkginfo_to_metadata(egginfo_path, egginfo_path)
+            with open(pkginfo_path, "r", encoding="utf-8") as inp:
+                print("\n\n\n~-------------------------~\nBefore wheel conversion:\n")
+                print(inp.read())
             os.mkdir(distinfo_path)
         else:
             # .egg-info is a directory
             pkginfo_path = os.path.join(egginfo_path, "PKG-INFO")
+            with open(pkginfo_path, "r", encoding="utf-8") as inp:
+                print("\n\n\n~-------------------------~\nBefore wheel conversion:\n")
+                print(inp.read())
             pkg_info = pkginfo_to_metadata(egginfo_path, pkginfo_path)

             # ignore common egg metadata that is useless to wheel
@@ -537,6 +543,10 @@
         with open(pkg_info_path, "w", encoding="utf-8") as out:
             Generator(out, mangle_from_=False, maxheaderlen=0).flatten(pkg_info)

+        with open(pkg_info_path, "r", encoding="utf-8") as out:
+            print("\n\n\n~-------------------------~\nAfter wheel conversion:\n")
+            print(out.read())
+
         for license_path in self.license_paths:
             filename = os.path.basename(license_path)
             shutil.copy(license_path, os.path.join(distinfo_path, filename))
EOF

patch -u .venv/lib/python3.10/site-packages/wheel/bdist_wheel.py -i investigation.patch
.venv/bin/pyproject-build --no-isolation
    

    The following is part of the output of the experiment:
    

    # ...
Copying helloworld.egg-info to build/bdist.linux-x86_64/wheel/helloworld-42-py3.10.egg-info
running install_scripts



~-------------------------~
Before wheel conversion:

Metadata-Version: 2.1
Name: helloworld
Version: 42
Maintainer-email: "Nicolai A. B. Riis" <nabr@dtu.dk>, "Jakob S. Jørgensen" <jakj@dtu.dk>, "Amal M. Alghamdi" <amaal@dtu.dk>




~-------------------------~
After wheel conversion:

Metadata-Version: 2.1
Name: helloworld
Version: 42
Maintainer-email: =?utf-8?b?Ik5pY29sYWkgQS4gQi4gUmlpcyIgPG5hYnJAZHR1LmRrPiwgIkpha29iIFMuIErDuHJnZW5zZW4iIDxqYWtqQGR0dS5kaz4sICJBbWFsIE0uIEFsZ2hhbWRpIiA8YW1hYWxAZHR1LmRrPg==?=


creating build/bdist.linux-x86_64/wheel/helloworld-42.dist-info/WHEEL
# 

    

    The output does seems to indicate that the change in the encoding is indeed happening inside of pypa/wheel. So I think the next step is to report it to that repository (I plan to do it soon, but please feel free to beat me to that).
    

    

    Thanks for the investigation! Seems this was just reported for summary META data in the linked issue. Looks like these issues are very similar. If you have some more in depth info to add please feel free. Thanks again.
  6. abravalheri commented on Nov 9, 2022 
    @abravalheri
    abravalheri
    on Nov 9, 2022
    Contributor
    I proposed a fix to pypa/wheel, which seems to solve the problem (I might be wrong though...).
    

    For the time being I think the best is to close this issue in favour of pypa/wheel#488.
    

    We can revisit and reopen it in the future if something else needs to be changed in the setuptools code base.
  7. abravalheri
    closed this as completedon Nov 9, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Metadata
Metadata
Assignees
No one assigned
    Labels
    Needs TriageIssues that need to be evaluated for severity and status.bug
    Projects
    No projects
      Milestone
      No milestone
      Relationships
      None yet
      Development
      No branches or pull requests
      Issue actions