API Log pull, redirect | CURL and script challenges (automation)

Problem:

• I want to pull logs from the API using CURL
• The redirected URL that is currently returned must be subject to scripting before it can be used for example in a crontab job
• I want to pull the logs every 4hrs automatically via CURL write to a file and send to my Splunk server. I'm writing a Splunk TA and I can share it.

Explanation:

# Step 1: (issue curl) This works for me.
curl -X GET -H "X-Api-Key: abcdefghijklmnopqrstuvwxyz0123456789" https://api.nextdns.io/profiles/abc123/logs/download > nextdns.log

# Step 2: Recieve a redirected URL (this works for me).
Found. Redirecting to https://api.nextdns.io/logs/abc123/abcdefghijklmnopqrstuvwxyz0123456789abcdefghijklmnopqrstuvwxyz0123456789abcdefghijklmnopqrstuvwxyz0123456789

# Step 3: (use updated URL for curl) This works for me.
curl -X GET -H "X-Api-Key: abcdefghijklmnopqrstuvwxyz0123456789" https://api.nextdns.io/logs/abc123/abcdefghijklmnopqrstuvwxyz0123456789abcdefghijklmnopqrstuvwxyz0123456789abcdefghijklmnopqrstuvwxyz0123456789 > nextdns.log
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 16.6M    0 16.6M    0     0  3936k      0 --:--:--  0:00:04 --:--:-- 3936k

# Step 4: cron job. This is unamanagable without scripting
crontab -e
27 * * * * bash /opt/remote-logs/nextdns/curl-script.sh

Copy

Request:

• How can the process of generating a URL be easier so that it can automatically be set to ingest?
• The redirect makes this not possible unless I regex out the new URL, insert that to a variable and then read that in each time, this is impractical.