How can update data for only one country not planet??

Hi,
I want to use overpass API with data related to my country not planet.
but I don't know how can I update that?
I think this is an update url :
http://download.geofabrik.de/asia/japan-updates/
but I don't know how can I find my current replication id?!!!
for using to this command:
nohup $EXEC_DIR/bin/fetch_osc.sh my_replication_id "https://download.geofabrik.de/asia/japan-updates" "$REPLICATE_DIR" &

but when i use above command, I get this error:
gzip: stdin: unexpected end of file
https://download.geofabrik.de/asia/japan-updates/003/950/542.osc.gz:
2020-03-27 15:24:12 ERROR 404: Not Found.

Thanks a lot for your help

Can you please follow up on help.openstreetmap.org? Frederik from Geofabrik can explain to you in more detail how they set up the numbering of their diff files, which would directly affect the replication ids as well, i.e. they're using some own numbering schema for their files which is different from what you see on planet.openstreetmap.org. By the way, the same logic would also apply to osm2pgsql so this isn't in any way Overpass specific.

As this is not a bug report or enhancement request, can you please close this issue? Thanks.

Can you please follow up on help.openstreetmap.org? Frederik from Geofabrik can explain to you in more detail how they set up the numbering of their diff files, which would directly affect the replication ids as well, i.e. they're using some own numbering schema for their files which is different from what you see on planet.openstreetmap.org. By the way, the same logic would also apply to osm2pgsql so this isn't in any way Overpass specific.

As this is not a bug report or enhancement request, can you please close this issue? Thanks.

Ok, I thought this issue is related to overpass, so I posted it here, but if you think I should ask my question another place it's Ok.
Any way thank you for your explaination

The process itself is pretty much the same as with a full planet: you start with the planet file and then apply all diff updates. In case of Geofabrik, you need to start with some extract for Japan, then set the replication id so it matches the diff files in the japan-updates folder. This is where you need to get more details from Geofabrik to find out which would be the right diff file to apply for a given Japan extract.

The process itself is pretty much the same as with a full planet: you start with the planet file and then apply all diff updates. In case of Geofabrik, you need to start with some extract for Japan, then set the replication id so it matches the diff files in the japan-updates folder. This is where you need to get more details from Geofabrik to find out which would be the right diff file to apply for a given Japan extract.

That's right. I searched and found that these addresses are based on daily updates:
https://download.geofabrik.de/asia/japan-updates
I changed some points in openstreetmap.org and expect to see them after getting diffs.
after running the following commands:

nohup $EXEC_DIR/bin/fetch_osc.sh 2562 "https://download.geofabrik.de/asia/japan-updates" "$REPLICATE_DIR" &

nohup $EXEC_DIR/bin/apply_osc_to_db.sh "$REPLICATE_DIR" 2562 --meta=yes &

I wanna to know that when the process is finished?
Should I change current_replication_id 2562 every day and run again the command?

apply_osc_to_db.sh usually runs as background job and fetches new data every 5 seconds (which is a bit pointless for a daily update). Best is to edit this file and increase the update interval to maybe 3600 seconds. If you don't want to run the script to run all the time, you also need to edit the loop in there and make the script terminate after some time.

Once you have the correct replicate id in a file called replicate_id, you can use auto instead of 2562 and the script will automatically pick up the latest value in the replicate_id file,

Thank you, the problem is the file replicate_id doesn't exists in db folder, so I think it should be created by myself, for example with getting this number from here:
http://download.geofabrik.de/asia/japan-updates/state.txt
and save it to replicate_id file.
and change the update interval in apply_osc_to_db.sh to for example every day (86400 seconds)
right?

replicate_id will be written here https://github.com/drolbr/Overpass-API/blob/master/src/bin/apply_osc_to_db.sh#L145 after the first successful update. You can of course create that file manually and put in the correct value matching the file http://download.geofabrik.de/asia/japan-latest.osm.pbf you've used for the initial database load. Remember that you have to start with such an extract as a first step to populate your db, only applying diffs from japan-updates will not work.

I'm not sure Geofabrik publishes the value for their japan-latest.osm.obf file. Maybe you can try osmium fileinfo -e to see if any of the header fields include this value, otherwise you need to ask Geofabrik.

sorry, I didn't understand what do you mean!
I downloaded this file http://download.geofabrik.de/asia/japan-latest.osm.pbf about a month ago
and made db folder and run the overpass API service on my server, and now I wanna to update the database with instrunctions said above
Is there a problem in the process done?

This looks ok to me. The only thing you need to make sure is that the replicate_id matches your japan-latest.osm.pbf file. If as an example your japan-latest is dated Feb 01, and you apply diffs starting with Feb 03, you might miss some objects and your database will be inconsistent/corrupt afterwards.

yes, you're right, I made a mistake!
I don't know how can I find replication_id from the last japan-latest.osm.bz2 file. where this number is writen?

I don't know how can I find replication_id from the last japan-latest.osm.bz2 file. where this number is writen?

I don't know either, that's why I suggested to ask Frederik from Geofabrik on help.openstreetmap.org

I asked him here
https://help.openstreetmap.org/questions/73794/how-can-update-data-for-only-one-country-not-planet-for-overpass-api

Thank you so much for your help and useful information

You can try the osmium tool (https://osmcode.org/osmium-tool/), which prints out all relevant osmosis_replication_* parameters.

osmium fileinfo germany-latest.osm.pbf

File:
  Name: germany-latest.osm.pbf
  Format: PBF
  Compression: none
  Size: 3176665308
Header:
  Bounding boxes:
    (5.864417,47.26543,15.05078,55.14777)
  With history: no
  Options:
    generator=osmium/1.5.1
    osmosis_replication_base_url=http://download.geofabrik.de/europe/germany-updates
    osmosis_replication_sequence_number=1494
    osmosis_replication_timestamp=2017-04-21T20:43:02Z
    pbf_dense_nodes=true
    timestamp=2017-04-21T20:43:02Z

Navigation Menu

Repository navigation

How can update data for only one country not planet?? #565

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

How can update data for only one country not planet?? #565

Description

Activity

mmd-osm commented on Mar 27, 2020

sogand145 commented on Mar 27, 2020

mmd-osm commented on Mar 27, 2020

sogand145 commented on Mar 27, 2020

mmd-osm commented on Mar 27, 2020

sogand145 commented on Mar 27, 2020

mmd-osm commented on Mar 27, 2020

sogand145 commented on Mar 27, 2020

mmd-osm commented on Mar 27, 2020

sogand145 commented on Mar 27, 2020

mmd-osm commented on Mar 27, 2020

sogand145 commented on Mar 27, 2020

mmd-osm commented on Mar 28, 2020

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions