Skip to content

Fixed ms2rescore bugs - #36

Merged
ypriverol merged 3 commits into
bigbio:devfrom
daichengxin:dev
Nov 29, 2024
Merged

ypriverol merged 3 commits into
bigbio:devfrom
daichengxin:dev

Conversation

@daichengxin

@daichengxin daichengxin commented Nov 28, 2024 •

Copy link
Copy Markdown
Collaborator

PR Type

enhancement, bug_fix


Description

  • Enhanced the PSM conversion process by adding handling for the 'q-value' meta value.
  • Introduced a new IDXMLReaderPatch class to improve idXML file reading, allowing for better PSM parsing with additional metadata and rescoring features.
  • Implemented logging to warn about invalid PSMs that lack search engine features, improving error handling and debugging.

Changes walkthrough 📝

Relevant files
Enhancement
psm_conversion.py
Enhance PSM conversion with 'q-value' handling                     

quantmsutils/psm/psm_conversion.py

  • Added handling for 'q-value' meta value in PSM conversion.
+2/-0     
ms2rescore.py
Enhance idXML reading and PSM parsing with metadata           

quantmsutils/rescoring/ms2rescore.py

  • Introduced IDXMLReaderPatch class for improved idXML reading.
  • Implemented PSM parsing with additional metadata and rescoring
    features.
  • Added logging for invalid PSMs without search engine features.
  • +100/-2 

    💡 PR-Agent usage: Comment /help "your question" on any pull request to receive relevant information

    @coderabbitai

    coderabbitai Bot commented Nov 28, 2024

    Copy link
    Copy Markdown
    Contributor

    Important

    Review skipped

    Auto reviews are disabled on base/target branches other than the default branch.

    Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

    You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.


    Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

    ❤️ Share
    🪧 Tips

    Chat

    There are 3 ways to chat with CodeRabbit:

    • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
      • I pushed a fix in commit <commit_id>, please review it.
      • Generate unit testing code for this file.
      • Open a follow-up GitHub issue for this discussion.
    • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
      • @coderabbitai generate unit testing code for this file.
      • @coderabbitai modularize this function.
    • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
      • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
      • @coderabbitai read src/utils.ts and generate unit testing code.
      • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
      • @coderabbitai help me debug CodeRabbit configuration file.

    Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

    CodeRabbit Commands (Invoked using PR comments)

    • @coderabbitai pause to pause the reviews on a PR.
    • @coderabbitai resume to resume the paused reviews.
    • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
    • @coderabbitai full review to do a full review from scratch and review all the files again.
    • @coderabbitai summary to regenerate the summary of the PR.
    • @coderabbitai resolve resolve all the CodeRabbit review comments.
    • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
    • @coderabbitai help to get help.

    Other keywords and placeholders

    • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
    • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
    • Add @coderabbitai anywhere in the PR title to generate the title automatically.

    CodeRabbit Configuration File (.coderabbit.yaml)

    • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
    • Please see the configuration documentation for more information.
    • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

    Documentation and Community

    • Visit our Documentation for detailed information on how to use CodeRabbit.
    • Join our Discord Community to get help, request features, and share feedback.
    • Follow us on X/Twitter for updates and announcements.

    @qodo-code-review

    Copy link
    Copy Markdown

    PR-Agent was enabled for this repository. To continue using it, please link your git user with your CodiumAI identity here.

    PR Reviewer Guide 🔍

    Here are some key observations to aid the review process:

    ⏱️ Estimated effort to review: 3 🔵🔵🔵⚪⚪
    🧪 No relevant tests
    🔒 No security concerns identified
    ⚡ Recommended focus areas for review

    Error Handling
    The print statement in error handling path should be replaced with proper logging. Also consider providing more context about which features are missing.

    Code Smell
    The type check print statement seems like debugging code that was left in and should be removed

    Potential Bug
    The rescoring_features dictionary is populated with float conversions without any error handling for invalid number formats

    @qodo-code-review

    qodo-code-review Bot commented Nov 28, 2024 •

    Copy link
    Copy Markdown

    PR-Agent was enabled for this repository. To continue using it, please link your git user with your CodiumAI identity here.

    PR Code Suggestions ✨

    Explore these optional code suggestions:

    CategorySuggestion                                                                                                                                    Score
    Possible issue
    Add error handling for float conversion of feature values to prevent runtime errors

    The rescoring_features dictionary is populated with unchecked float conversions that
    could raise ValueError. Add error handling for the float conversion.

    quantmsutils/rescoring/ms2rescore.py [86]

    -rescoring_features[key] = float(peptide_hit.getMetaValue(key))
    +try:
    +    rescoring_features[key] = float(peptide_hit.getMetaValue(key))
    +except ValueError:
    +    logging.warning(f"Could not convert feature {key} to float")
    +    return None
    • Apply this suggestion
    Suggestion importance[1-10]: 7

    Why: The suggestion addresses a potential runtime error by adding try-except handling for float conversion, which could prevent crashes when dealing with invalid feature values.

    7
    General
    ✅ Improve error handling for missing features with proper logging instead of debug printing
    Suggestion Impact:The commit removed the debug print statement for missing features, which aligns with the suggestion to improve error handling. However, it did not implement logging as suggested.

    code diff:

                 if not feature:
    -                print(type(feature))
                     return None

    The feature existence check should handle the return value more gracefully. Instead
    of printing the type and returning None, implement proper error handling or logging
    for missing features.

    quantmsutils/rescoring/ms2rescore.py [81-86]

     feature = peptide_hit.metaValueExists(key)
     if not feature:
    -    print(type(feature))
    +    logging.debug(f"Missing required feature: {key}")
         return None
     else:
         rescoring_features[key] = float(peptide_hit.getMetaValue(key))
    • Apply this suggestion
    Suggestion importance[1-10]: 6

    Why: The suggestion improves error handling by replacing a debug print with proper logging, making the code more maintainable and easier to debug. While not critical, it's a good practice enhancement.

    6

    💡 Need additional feedback ? start a PR chat

    @ypriverol ypriverol left a comment

    Copy link
    Copy Markdown
    Contributor

    Choose a reason for hiding this comment

    The reason will be displayed to describe this comment to others. Learn more.

    @daichengxin can you provide a test case for this? Basically take the test and remove for one PSM the scores.

    @ypriverol
    ypriverol requested a review from jpfeuffer November 28, 2024 15:17
    @ypriverol

    Copy link
    Copy Markdown
    Contributor

    @jpfeuffer can you also take a look here in case Im missing something.

    peptide_id_metadata = {
    "idxml:score_type": str(peptide_id.getScoreType()),
    "idxml:higher_score_better": str(peptide_id.isHigherScoreBetter()),
    "idxml:significance_threshold": str(peptide_id.getSignificanceThreshold()),

    Copy link
    Copy Markdown
    Contributor

    Choose a reason for hiding this comment

    The reason will be displayed to describe this comment to others. Learn more.

    we probably don't use the significance threshold

    @ypriverol ypriverol Nov 28, 2024 •

    Copy link
    Copy Markdown
    Contributor

    Choose a reason for hiding this comment

    The reason will be displayed to describe this comment to others. Learn more.

    what do you mean? I think this logic is in the original ms2rescore from @jonasscheid, no @daichengxin? How is it used in ms2rescore?

    @ypriverol ypriverol Nov 28, 2024 •

    Copy link
    Copy Markdown
    Contributor

    Choose a reason for hiding this comment

    The reason will be displayed to describe this comment to others. Learn more.

    @jpfeuffer is this always annotated: peptide_id.getSignificanceThreshold()?

    Copy link
    Copy Markdown

    Choose a reason for hiding this comment

    The reason will be displayed to describe this comment to others. Learn more.

    Afaik MS2Rescore computes the q-value before rescoring and IdXMLWriter puts it as a metaValue in the PeptideHit obj. Maybe you can just get that score and probably you don't need to to any extra steps in that regard

    Copy link
    Copy Markdown
    Collaborator Author

    Choose a reason for hiding this comment

    The reason will be displayed to describe this comment to others. Learn more.

    what do you mean? I think this logic is in the original ms2rescore from @jonasscheid, no @daichengxin? How is it used in ms2rescore?

    Yes. This logic is in the original ms2rescore. I only reconstruct parse psm function to skip invalid psm without search engines score features.

    @ypriverol ypriverol Nov 29, 2024 •

    Copy link
    Copy Markdown
    Contributor

    Choose a reason for hiding this comment

    The reason will be displayed to describe this comment to others. Learn more.

    @jonasscheid This error comes from MSGF+, which produces some PSMs without search engine scores. bigbio/quantms#447

    In the new version of OpenMS @timosachsenberg has introduced a patch OpenMS/OpenMS#7713. Im wondering if we need to also remove the PSMs with the latest dummy values:

    hit->setMetaValue("MeanErrorAll", 0.0);
    hit->setMetaValue("StdevErrorAll", 0.0);
    hit->setMetaValue("MeanErrorTop7", 0.0);
    hit->setMetaValue("StdevErrorTop7", 0.0);
    hit->setMetaValue("MeanRelErrorAll", 0.0);
    hit->setMetaValue("StdevRelErrorAll", 0.0);
    hit->setMetaValue("MeanRelErrorTop7", 0.0);
    hit->setMetaValue("StdevRelErrorTop7", 0.0);
    

    @jpfeuffer @timosachsenberg @daichengxin @jonasscheid

    Copy link
    Copy Markdown
    Contributor

    Choose a reason for hiding this comment

    The reason will be displayed to describe this comment to others. Learn more.

    I would keep them an evaluate

    Copy link
    Copy Markdown
    Collaborator Author

    Choose a reason for hiding this comment

    The reason will be displayed to describe this comment to others. Learn more.

    I vote keep if these dummy values have not effect. Are these PSMs likely to appear in the final results or are they filtered out in subsequent steps?

    Copy link
    Copy Markdown
    Contributor

    Choose a reason for hiding this comment

    The reason will be displayed to describe this comment to others. Learn more.

    If they are of general low quality I think ms2rescore and percolator will assign a bad score so they get filtered out.
    So no problem there. On the other had good hits (e.g., with good denovo or raw score) can still survice the filtering even though the other scores are set to dummy values. I think this is better than just filtering them.

    Copy link
    Copy Markdown
    Contributor

    Choose a reason for hiding this comment

    The reason will be displayed to describe this comment to others. Learn more.

    But they will not be used, no? With no score, those PSMS will be noisy all over the place. I can guess they will receive some scores coming from ms2rescore because some new features are introduced, like PEPLEN, PredictedRT, MS2PIP, etc., but the original search engine scores will be gone. In the downstream analysis, this will also be strange because you have PSMs with no Scores.

    BTW, @timosachsenberg the version from MSGF+ that OpenMS is using is MS-GF+ Release (v2023.01.12) (12 January 2023), I re-run the dataset with the latest version of MSGF+ here the mziD: bigbio/quantms#447 (comment)

    @jpfeuffer

    Copy link
    Copy Markdown
    Contributor

    Looks fine. I also never used the significance threshold.

    @ypriverol
    ypriverol merged commit 6d285fc into bigbio:dev Nov 29, 2024
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

    Projects

    None yet

    Development

    Successfully merging this pull request may close these issues.

    5 participants