Implement Glue Catalog #249

Activity

1. liurenjie1024

   added this to the 0.3.0 Release milestone on Mar 11, 2024
2. liurenjie1024

   added this to iceberg-ruston Mar 11, 2024
3. liurenjie1024

   removed this from the 0.3.0 Release milestone on Mar 11, 2024
4. liurenjie1024

   changed the title [-]Glue Catalog[/-] [+]Implement Glue Catalog[/+] on Mar 11, 2024
5. liurenjie1024 commented on Mar 11, 2024

   

   liurenjie1024

   on Mar 11, 2024

   ContributorAuthor

   More actions

   Blocked by #250
6. marvinlanhenke commented on Mar 22, 2024

   

   marvinlanhenke

   on Mar 22, 2024

   Last edited by marvinlanhenke

   Contributor

   More actions

   @liurenjie1024
   ... I started prototyping on this, here is what I'm trying to do:

   Tasks:

   • Setup basic structure + test infra
   • Implement namespace operations
   • Implement table operations
7. liurenjie1024 commented on Mar 22, 2024

   

   liurenjie1024

   on Mar 22, 2024

   ContributorAuthor

   More actions

   Hi, @marvinlanhenke Thanks for this. I'm curious how do you plan to setup test infra for glue? It seems that there is no docker for glue?
8. marvinlanhenke commented on Mar 22, 2024

   

   marvinlanhenke

   on Mar 22, 2024

   Last edited by marvinlanhenke

   Contributor

   More actions

   I'm curious how do you plan to setup test infra for glue? It seems that there is no docker for glue?

   I looked at the pyiceberg tests which utilize moto.
   Moto provides a docker-compose to mock most of AWS Services / I tested this with a quick prototype and it seems to work fine in conjunction with the AWS SDK Rust.

   https://github.com/apache/iceberg-python/blob/main/tests/catalog/test_glue.py#L23
   https://docs.getmoto.org/en/latest/docs/server_mode.html#run-using-docker
9. liurenjie1024 commented on Mar 22, 2024

   

   liurenjie1024

   on Mar 22, 2024

   ContributorAuthor

   More actions

   I'm curious how do you plan to setup test infra for glue? It seems that there is no docker for glue?

   I looked at the pyiceberg tests which utilize moto. Moto provides a docker-compose to mock most of AWS Services / I tested this with a quick prototype and it seems to work fine in conjunction with the AWS SDK Rust.

   https://github.com/apache/iceberg-python/blob/main/tests/catalog/test_glue.py#L23 https://docs.getmoto.org/en/latest/docs/server_mode.html#run-using-docker

   That's great! Thanks!
10. liurenjie1024

    moved this to In Progress in iceberg-ruston Mar 26, 2024
11. liurenjie1024

    added this to the 0.3.0 Release milestone on Mar 26, 2024
12. liurenjie1024

    closed this as completedin #314on Apr 21, 2024
13. github-project-automation

    moved this from In Progress to Done in iceberg-ruston Apr 21, 2024