How do I add a GitHub Package repository when executing spark-submit --repositories?

I’m trying to consume in Spark a package published to GitHub Packages here. This package is meant to be used in Spark with a command such as this:

spark-submit --master "local[*]" \
  --class com.target.data_validator.Main \
  --packages com.target:data-validator_2.11:0.13.2 \
  --repositories "https://${GITHUB_USER}:${GITHUB_PACKAGES_TOKEN}@maven.pkg.github.com/target/data-validator" \
  empty.file \
  --config local_validators.yaml \
  --jsonReport report.json \
  --htmlReport report.html

When I run this, I get the ol’ not found error:

[unresolved dependency: com.target#data-validator_2.11;0.13.2: not found]

Looking at the URLs that Spark tried for resolution, I can see the repository I’ve specified in the command, along with the correct credentials specified above as variables. If I copy-paste the URL that spark-submit claims to have tried, it works in a browser, with wget, and with curl. I speculate that Spark is having trouble handling the credentials or the redirect(s) that GHP provides before giving the final data URL.

I also tried creating an Ivy settings file and passing it to the spark-submit command with --conf spark.jars.ivySettings=dv-ivy.xml:

<ivysettings>
	<settings defaultResolver="ghp-dv"/>
	<credentials 
		host="maven.pkg.github.com"
		username="${GITHUB_USER}"
		passwd="${GITHUB_PACKAGES_TOKEN}"/>
		<!-- real credentials hardcoded -->
	<resolvers>
		<ibiblio
			name="ghp-dv"
			m2compatible="true"
			root="https://maven.pkg.github.com/target/data-validator"/>
	</resolvers>
</ivysettings>

The tried URLs are the same and also are valid in browser, etc. I’m pretty sure the problem is in Spark’s handling of credentials and redirects.

How can I properly configure Spark to resolve and retrieve the package from GitHub Packages with authentication currently required?

Side note: I am using Spark 2.3.1 and cannot use anything newer right now for reasons outside of my control.

(crossposted on Stack Overflow as How do I add a GitHub Package repository when executing spark-submit --repositories?)

Related questions that didn’t provide an answer:

I’ve found that the resolver works if the realm is specified. I consider this a stopgap solution, as this is really not ergonomic since a user would have to duplicate their entirely system-wide Ivy settings. This is probably acceptable for one-off and single-user projects but on a Spark cluster, it’s a really sharp edge.

<ivysettings>
	<settings defaultResolver="ghp-dv"/>
	<credentials 
		host="maven.pkg.github.com"
		realm="GitHub Package Registry"
		username="${GITHUB_USER}"
		passwd="${GITHUB_PACKAGES_TOKEN}"/>
		<!-- real credentials hardcoded -->
	<resolvers>
		<ibiblio
			name="ghp-dv"
			m2compatible="true"
			root="https://maven.pkg.github.com/target/data-validator"/>
	</resolvers>
</ivysettings>

So, I find myself longing for unauthenticated downloads like everyone else, or a solution that doesn’t require specifying spark.jars.ivySettings and managing a configuration duplicated from the cluster configuration.