KDOC 84: いろんなサイトでOPTIONSメソッドを送ったときのレスポンスヘッダーを見る

この文書のステータス

  • 作成
    • 2024-02-10 貴島
  • レビュー
    • 2024-02-12 貴島

概要

さまざまなWebサイトのレスポンスをcurlで見ているうち、微妙な違いがあること、知らないヘッダーが多いのに気づいた。適当に見たものをメモしておく。

より差異を見るために、CORSのプリフライト風のリクエストで確認した。CORSで制御しているサーバであれば、OPTIONSメソッドとAccess-Control-Request-Methodで指定したアクションを渡すと、許可状況を返す。リソースへのアクセスをCORS以外の方法で制御しているとOPTIONSメソッドを使うことはおそらくないので、OPTIONSアクション自体を許可してない場合もある。

見る

Google

curl -I -H "Access-Control-Request-Method: GET" -X OPTIONS https://www.google.com
HTTP/2 405
allow: GET, HEAD
date: Sat, 10 Feb 2024 04:19:05 GMT
content-type: text/html; charset=UTF-8
server: gws
content-length: 1592
x-xss-protection: 0
x-frame-options: SAMEORIGIN
alt-svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000

コメント。

YouTube

curl -I -H "Access-Control-Request-Method: GET" -X OPTIONS https://youtube.com
HTTP/2 405
content-type: text/html; charset=UTF-8
referrer-policy: no-referrer
content-length: 1592
date: Sat, 10 Feb 2024 04:19:01 GMT
alt-svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000

コメント。

Yahoo.co.jp

curl -I -H "Access-Control-Request-Method: GET" -X OPTIONS https://www.yahoo.co.jp/
HTTP/2 200
server: nginx
date: Sat, 10 Feb 2024 04:18:59 GMT
content-type: text/html; charset=utf-8
content-length: 8
allow: GET,HEAD
etag: W/"8-ZRAf8oNBS3Bjb/SU2GYZCmbtmXg"
set-cookie: B=au4d8p5isdu9j&b=3&s=uk; expires=Tue, 10-Feb-2026 04:18:59 GMT; path=/; domain=.yahoo.co.jp; Secure
traceresponse: 00-37df146f1bdfb51a998b43311d35e2c3-28c16e2113872761-01
vary: Accept-Encoding
x-dt-tracestate: 945f54dd-87dd099b@dt
x-vcap-request-id: 86f2560f-5ad6-4208-5edf-6bd9f55d2ed8
x-xss-protection: 1; mode=block
age: 0
accept-ch: Sec-CH-UA-Full-Version-List, Sec-CH-UA-Model, Sec-CH-UA-Platform-Version, Sec-CH-UA-Arch
permissions-policy: ch-ua-full-version-list=*, ch-ua-model=*, ch-ua-platform-version=*, ch-ua-arch=*
permissions-policy: unload=()

  • 項目が多い
  • nginx
  • charsetの値は小文字
  • HEADに対応している

X

https://x.com にリクエストするとtwitter.comへリダイレクトがかかる。

curl -I -H "Access-Control-Request-Method: GET" -X OPTIONS https://twitter.com
HTTP/2 200
date: Sat, 10 Feb 2024 04:18:41 GMT
perf: 7469935968
expiry: Tue, 31 Mar 1981 05:00:00 GMT
pragma: no-cache
server: tsa_m
set-cookie: guest_id_marketing=v1%3A170753872176677350; Max-Age=63072000; Expires=Mon, 09 Feb 2026 04:18:41 GMT; Path=/; Domain=.twitter.com; Secure; SameSite=None
set-cookie: guest_id_ads=v1%3A170753872176677350; Max-Age=63072000; Expires=Mon, 09 Feb 2026 04:18:41 GMT; Path=/; Domain=.twitter.com; Secure; SameSite=None
set-cookie: personalization_id="v1_+xD3v4ygrkSbj4kwmOVewg=="; Max-Age=63072000; Expires=Mon, 09 Feb 2026 04:18:41 GMT; Path=/; Domain=.twitter.com; Secure; SameSite=None
set-cookie: guest_id=v1%3A170753872176677350; Max-Age=63072000; Expires=Mon, 09 Feb 2026 04:18:41 GMT; Path=/; Domain=.twitter.com; Secure; SameSite=None
set-cookie: ct0=; Max-Age=-1707538720; Expires=Thu, 01 Jan 1970 00:00:01 GMT; Path=/; Domain=.twitter.com; Secure; SameSite=Lax
content-type: text/html; charset=utf-8
x-powered-by: Express
cache-control: no-cache, no-store, must-revalidate, pre-check=0, post-check=0
last-modified: Sat, 10 Feb 2024 04:18:41 GMT
x-frame-options: DENY
x-transaction-id: 6411e2e240eddb17
x-xss-protection: 0
x-content-type-options: nosniff
content-security-policy: connect-src 'self' blob: https://api.x.ai https://api.x.com https://*.pscp.tv https://*.video.pscp.tv https://*.twimg.com https://api.twitter.com https://api.x.com https://api-stream.twitter.com https://api-stream.x.com https://ads-api.twitter.com https://ads-api.x.com https://aa.twitter.com https://aa.x.com https://caps.twitter.com https://caps.x.com https://pay.twitter.com https://pay.x.com https://sentry.io https://ton.twitter.com https://ton.x.com https://ton-staging.atla.twitter.com https://ton-staging.atla.x.com https://ton-staging.pdxa.twitter.com https://ton-staging.pdxa.x.com https://twitter.com https://x.com https://upload.twitter.com https://upload.x.com https://www.google-analytics.com https://accounts.google.com/gsi/status https://accounts.google.com/gsi/log https://checkoutshopper-live.adyen.com wss://*.pscp.tv https://vmap.snappytv.com https://vmapstage.snappytv.com https://vmaprel.snappytv.com https://vmap.grabyo.com https://dhdsnappytv-vh.akamaihd.net https://pdhdsnappytv-vh.akamaihd.net https://mdhdsnappytv-vh.akamaihd.net https://mdhdsnappytv-vh.akamaihd.net https://mpdhdsnappytv-vh.akamaihd.net https://mmdhdsnappytv-vh.akamaihd.net https://mdhdsnappytv-vh.akamaihd.net https://mpdhdsnappytv-vh.akamaihd.net https://mmdhdsnappytv-vh.akamaihd.net https://dwo3ckksxlb0v.cloudfront.net https://media.riffsy.com https://*.giphy.com https://media.tenor.com https://c.tenor.com https://ads-twitter.com https://analytics.twitter.com https://analytics.x.com  ; default-src 'self'; form-action 'self' https://twitter.com https://*.twitter.com https://x.com https://*.x.com; font-src 'self' https://*.twimg.com; frame-src 'self' https://twitter.com https://x.com https://mobile.twitter.com https://mobile.x.com https://pay.twitter.com https://pay.x.com https://cards-frame.twitter.com https://accounts.google.com/ https://client-api.arkoselabs.com/ https://iframe.arkoselabs.com/ https://vaultjs.apideck.com/  https://recaptcha.net/recaptcha/ https://www.google.com/recaptcha/ https://www.gstatic.com/recaptcha/; img-src 'self' blob: data: https://*.cdn.twitter.com https://*.cdn.x.com https://ton.twitter.com https://ton.x.com https://*.twimg.com https://analytics.twitter.com https://analytics.x.com https://cm.g.doubleclick.net https://www.google-analytics.com https://maps.googleapis.com https://www.periscope.tv https://www.pscp.tv https://ads-twitter.com https://ads-api.twitter.com https://ads-api.x.com https://media.riffsy.com https://*.giphy.com https://media.tenor.com https://c.tenor.com https://*.pscp.tv https://*.periscope.tv https://prod-periscope-profile.s3-us-west-2.amazonaws.com https://platform-lookaside.fbsbx.com https://scontent.xx.fbcdn.net https://scontent-sea1-1.xx.fbcdn.net https://*.googleusercontent.com https://t.co/1/i/adsct; manifest-src 'self'; media-src 'self' blob: https://twitter.com https://x.com https://*.twimg.com https://*.vine.co https://*.pscp.tv https://*.video.pscp.tv https://dhdsnappytv-vh.akamaihd.net https://pdhdsnappytv-vh.akamaihd.net https://mdhdsnappytv-vh.akamaihd.net https://mdhdsnappytv-vh.akamaihd.net https://mpdhdsnappytv-vh.akamaihd.net https://mmdhdsnappytv-vh.akamaihd.net https://mdhdsnappytv-vh.akamaihd.net https://mpdhdsnappytv-vh.akamaihd.net https://mmdhdsnappytv-vh.akamaihd.net https://dwo3ckksxlb0v.cloudfront.net; object-src 'none'; script-src 'self' 'unsafe-inline' https://*.twimg.com https://recaptcha.net/recaptcha/ https://www.google.com/recaptcha/ https://www.gstatic.com/recaptcha/ https://client-api.arkoselabs.com/ https://www.google-analytics.com https://twitter.com https://x.com https://accounts.google.com/gsi/client https://appleid.cdn-apple.com/appleauth/static/jsapi/appleid/1/en_US/appleid.auth.js https://static.ads-twitter.com  'nonce-Yzg1ZTA0YmEtN2ZlNi00Mzc4LWI0N2MtZWJlYTViMDgwNWMx'; style-src 'self' 'unsafe-inline' https://accounts.google.com/gsi/style https://*.twimg.com; worker-src 'self' blob:; report-uri https://twitter.com/i/csp_report?a=O5RXE%3D%3D%3D&ro=false
strict-transport-security: max-age=631138519
cross-origin-opener-policy: same-origin-allow-popups
cross-origin-embedder-policy: unsafe-none
x-response-time: 119
x-connection-hash: 806661cfc51ee7c7cda80afc197b466cbbe72ae93be498085289fb353e4e7b73

メモ。

  • インラインでいろいろなサイトのコンテンツを表示するためか、許可URLが多い
  • serverはtsa_m。不明
  • レスポンス時間を独自ヘッダーにのせている
  • x-powered-byがExpress

GitHub

curl -I -H "Access-Control-Request-Method: GET" -X OPTIONS https://github.com/
HTTP/2 404
server: GitHub.com
date: Sat, 10 Feb 2024 04:18:32 GMT
content-type: text/html; charset=utf-8
content-length: 128674
strict-transport-security: max-age=31536000; includeSubdomains; preload
x-frame-options: deny
x-content-type-options: nosniff
x-xss-protection: 0
referrer-policy: origin-when-cross-origin, strict-origin-when-cross-origin
content-security-policy: default-src 'none'; base-uri 'self'; child-src github.com/assets-cdn/worker/ gist.github.com/assets-cdn/worker/; connect-src 'self' uploads.github.com www.githubstatus.com collector.github.com raw.githubusercontent.com api.github.com github-cloud.s3.amazonaws.com github-production-repository-file-5c1aeb.s3.amazonaws.com github-production-upload-manifest-file-7fdce7.s3.amazonaws.com github-production-user-asset-6210df.s3.amazonaws.com api.githubcopilot.com objects-origin.githubusercontent.com; font-src github.githubassets.com; form-action 'self' github.com gist.github.com objects-origin.githubusercontent.com; frame-ancestors 'none'; frame-src viewscreen.githubusercontent.com notebooks.githubusercontent.com; img-src 'self' data: github.githubassets.com media.githubusercontent.com camo.githubusercontent.com identicons.github.com avatars.githubusercontent.com github-cloud.s3.amazonaws.com objects.githubusercontent.com secured-user-images.githubusercontent.com/ user-images.githubusercontent.com/ private-user-images.githubusercontent.com opengraph.githubassets.com github-production-user-asset-6210df.s3.amazonaws.com customer-stories-feed.github.com spotlights-feed.github.com objects-origin.githubusercontent.com; manifest-src 'self'; media-src github.com user-images.githubusercontent.com/ secured-user-images.githubusercontent.com/ private-user-images.githubusercontent.com github-production-user-asset-6210df.s3.amazonaws.com gist.github.com; script-src github.githubassets.com; style-src 'unsafe-inline' github.githubassets.com; upgrade-insecure-requests; worker-src github.com/assets-cdn/worker/ gist.github.com/assets-cdn/worker/
vary: Accept-Encoding, Accept, X-Requested-With
x-github-request-id: B0EE:255506:53D993:5DD90B:65C6F918

  • 404エラー
  • WebサーバはOPTIONSを受け入れている。が、少なくともルートパスでは実装はない
  • content-security-policyがたくさんある
    • gist.github.com
    • upload.github.com
    • api.githubcopilot.com
    • s3

Amazon.com

curl -I -H "Access-Control-Request-Method: GET" -X GET https://www.amazon.com/
HTTP/2 503
content-type: text/html
server: Server
date: Sat, 10 Feb 2024 04:18:27 GMT
x-amz-rid: GZYGAQ9G8CWDXZ7H47FW
vary: Content-Type,Accept-Encoding,User-Agent
last-modified: Tue, 30 Jan 2024 23:36:50 GMT
etag: "a6f-610323c84a880"
accept-ranges: bytes
strict-transport-security: max-age=47474747; includeSubDomains; preload
x-cache: Error from cloudfront
via: 1.1 2ae17d68ad090fea921cea9935f8b4e4.cloudfront.net (CloudFront)
x-amz-cf-pop: NRT20-C3
alt-svc: h3=":443"; ma=86400
x-amz-cf-id: wA8l7JDrCSL14_9OWmg6U1k-_8nwnSuZkItvkWVaOkCI_lKvyWJ_mQ==

  • 503 Service Unavailable - HTTP | MDN。User-Agentがcurlだと503になるよう
  • serverはServer。適当な値っぽい。個人的には、動いているWebサーバが知れたところで運用や利用側にメリットはなく、セキュリティリスクしかないように見えるので、これがよいと考えている。が、多数派ではなさそう
  • x-amz-cf-系はCDN CloudFrontが返すヘッダー。NRT(成田)はエッジの場所を表すコード。たまたま空港コードだが、別に空港とは関係がなさそう
curl -I -A "xxxx" -H "Access-Control-Request-Method: GET" -X GET https://www.amazon.com/
HTTP/2 200
content-type: text/html;charset=UTF-8
server: Server
date: Sat, 10 Feb 2024 04:18:22 GMT
x-amz-rid: YACTCQE1QM03TY6MGBWY
set-cookie: session-id=142-6963675-6615257; Domain=.amazon.com; Expires=Sun, 09-Feb-2025 04:18:22 GMT; Path=/; Secure
set-cookie: session-id-time=2082787201l; Domain=.amazon.com; Expires=Sun, 09-Feb-2025 04:18:22 GMT; Path=/; Secure
set-cookie: i18n-prefs=USD; Domain=.amazon.com; Expires=Sun, 09-Feb-2025 04:18:22 GMT; Path=/
set-cookie: sp-cdn="L5Z9:JP"; Version=1; Domain=.amazon.com; Max-Age=31536000; Expires=Sun, 09-Feb-2025 04:18:22 GMT; Path=/; Secure; HttpOnly
set-cookie: skin=noskin; path=/; domain=.amazon.com
accept-ch: ect,rtt,downlink,device-memory,sec-ch-device-memory,viewport-width,sec-ch-viewport-width,dpr,sec-ch-dpr,sec-ch-ua-platform,sec-ch-ua-platform-version
pragma: no-cache
x-xss-protection: 1;
content-security-policy: upgrade-insecure-requests;report-uri https://metrics.media-amazon.com/
content-security-policy-report-only: default-src 'self' blob: https: data: mediastream: 'unsafe-eval' 'unsafe-inline';report-uri https://metrics.media-amazon.com/
x-content-type-options: nosniff
content-language: en-US
content-encoding: gzip
expires: -1
x-ua-compatible: IE=edge
cache-control: no-cache
accept-ch-lifetime: 86400
strict-transport-security: max-age=47474747; includeSubDomains; preload
vary: Content-Type,Accept-Encoding,User-Agent
x-frame-options: SAMEORIGIN
x-cache: Miss from cloudfront
via: 1.1 33adaf636d9a8b17ab166777508ba07a.cloudfront.net (CloudFront)
x-amz-cf-pop: NRT20-C3
alt-svc: h3=":443"; ma=86400
x-amz-cf-id: 9MHMYlGivQUaXwugJ-xT0BenC2Sett1G6AxEZ8Bw0ETlZnyrv7fqtA==

  • User-Agentがcurl以外だと成功する
  • strict-transport-securityヘッダのmax-ageは秒数。
    • Strict-Transport-Security - HTTP | MDN。ウェブサイトがブラウザーにHTTPの代わりにHTTPSを用いて通信を行うよう指示するためのもの
    • max-age: 秒単位で、そのサイトに HTTPS だけで接続することをブラウザーが記憶する時間です。
    • 47474747秒 = 549日11時間25分47秒
    • 長い。なぜこの数値なのだろう

はてな

curl -I -H "Access-Control-Request-Method: GET" -X OPTIONS https://b.hatena.ne.jp/
HTTP/2 405
content-type: text/plain
content-length: 22
date: Sat, 10 Feb 2024 04:18:18 GMT
server: nginx
x-dispatch: Boston::Web::Public#process
x-cache: Error from cloudfront
via: 1.1 f1f4afba4268f1486380be4c4394d85c.cloudfront.net (CloudFront)
x-amz-cf-pop: NRT57-P4
x-amz-cf-id: ZcohLpRTN2vylUxFhH36I5aCoud-PAM7d4N6GeNkws7vp_b9fL1dag==

  • content-typeの値はtext/plain。あまり多くないパターン
  • 405 Method Not Allowed - HTTP | MDN
  • 独自リクエストヘッダーx-dispatchはボストンとなっているが、何がボストンなのだろう

楽天

curl -I -H "Access-Control-Request-Method: GET" -X OPTIONS https://www.rakuten.co.jp/
HTTP/2 200
server: Apache
pragma: no-cache
cache-control: private
x-content-type-options: nosniff
x-xss-protection: 1; mode=block
x-frame-options: DENY
content-type: text/html
date: Sat, 10 Feb 2024 04:18:11 GMT
set-cookie: Apache=ddf909d5.610ff551dbbdf; path=/; expires=Sun, 09-Feb-25 04:18:10 GMT
set-cookie: wPzd=lng%3DNA%3Acnt%3DJP; expires=Sun, 09-Feb-2025 04:18:10 GMT; path=/; domain=www.rakuten.co.jp
vary: User-Agent

まとめ

  • serverヘッダーはけっこう違う。Webサーバはいろいろ使われている。あるいは適当なのを入れてるパターンもある
  • 微妙な表記のゆれがある。大文字小文字とか、MIMEの違いとか
    • Content-Typeはtext/htmlが多数派だった
  • CloudFrontを使っているケースが多い
  • HTTP/2に対応している(AWSのロードバランサか)
  • curlを弾くケースがある

関連