My wavefront proxy is running in a docker container and is continually reporting:
Jun 08, 2016 6:27:18 PM com.wavefront.agent.QueuedAgentService handleTaskRetry
WARNING: Cannot post push data result to Wavefront servers. Will enqueue and retry later: java.lang.RuntimeException: Server error:
407
Jun 08, 2016 6:27:18 PM com.wavefront.agent.AbstractAgent fetchConfig
INFO: fetching configuration from server at: https://try.wavefront.com/api/
Jun 08, 2016 6:27:20 PM com.wavefront.agent.QueuedAgentService$3 run
WARNING: [RETRY THREAD 0] cannot submit data to Wavefront servers. Will re-attempt later
java.lang.RuntimeException: Server error: 407
at com.wavefront.agent.QueuedAgentService.parsePostingResponse(QueuedAgentService.java:470)
at com.wavefront.agent.QueuedAgentService.access$700(QueuedAgentService.java:65)
at com.wavefront.agent.QueuedAgentService$PostPushDataResultTask.execute(QueuedAgentService.java:557)
at com.wavefront.agent.QueuedAgentService$3.run(QueuedAgentService.java:174)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
The API token and URL seem to be set correctly in the config file (/opt/wavefront/wavefront-proxy/conf/wavefront.conf). Any ideas?
Hi Stephan,
Error 407 is what you would normally get if your http traffic is routed through a proxy server and it requires authentication - if bypassing the proxy is not an option, could you please try setting the HTTP_PROXY environment variable and restart the container? The following template could be used to configure the proxy with authentication:
export HTTP_PROXY="http://{username}:{password}@{proxy_host}:{proxy_port}"
Please let us know if it helps!
-Vasily
We're not using a proxy. I can run a shell in the same docker container and make requests to https://metrics.wavefront.com/api using the same auth token as the proxy without issue.
That makes sense - in this case, could you please update the wavefront.conf file to point to metrics.wavefront.com instead of try.wavefront.com and restart the proxy? The default setting in the config points to the trial instance, but your account is actually set up on a production "metrics" cluster.
-Vasily
The config file says "server=https://metrics.wavefront.com/api". There's no mention of try.wavefront.com in the config, though that does appear in the logs.
I restarted the proxy *within* the container. I think there may be a race between writing the config file and starting the proxy when the docker container starts. It now reports that it's retrieving config from metrics.wavefront.com (instead of try.wavefront.com) and it draining its queue.
This turns out to be caused by a bug in the install script. I published a pull request with a fix: