Add HTTP.blocklist for denying requests by IP/host

[benubois]

Hello,

This adds a blocklist option, which raises HTTP::BlockedHostError if the request resolves to an IP or host which is blocked.

http = HTTP.blocklist(
  IPAddr.new("169.254.0.0/16"),                                # link-local
  "internal.example.com",                                      # by hostname
  deny: -> (address) { address.loopback? || address.private? } # proc to let the user decide if the address is safe
)

http.get("http://127.0.0.1/")   # raises HTTP::BlockedHostError
http.get("https://example.com") # allowed

This supports:

• IPAddr for IP addresses. CIDR is fine too like IPAddr.new("127.0.0.0/8")
• String for matching hostnames host and its subdomains (for example if you block internal.example.com, db.internal.example.com will also be blocked.
• deny: receives each resolved address as an IPAddr and blocks it by if it returns true

Also available as a constructor and per-request option:

HTTP::Client.new(blocklist: [IPAddr.new("10.0.0.0/8")])
HTTP.get(url, blocklist: { entries: [...], deny: ->(address) { address.loopback? } })
HTTP.blocklist([IPAddr.new("169.254.0.0/16"), IPAddr.new("127.0.0.1/8")])

Limitations

• A blocklist cannot be enforced through a proxy. The proxy resolves the target and makes the connection, so the addresses checked locally are not necessarily the ones reached. The risk here is probably limited to DNS rebinding

• No Happy Eyeballs for blocklisted requests. Since the address is validated up front the OS cannot try each one if multiple addresses are returned

Example Use Case

Currently Mastodon uses http, but does its own DNS resolution and blocking. With this feature built in to http, all of this could become something like

HTTP.blocklist(deny: PrivateAddressCheck.method(:private_address?))

Thanks!