Hi Doug

That's exactly what I though a while ago, actually my another project, couchcache, does something similar.

However I didn't feel secure mocking responses from service provider. One recent example is, I realised MapQuest Nominatim introduces an access key months after they did it.

Live tests reduce the value of their mocking counterpart in most cases. But the intermittent build failure is definitely something needs to fix.

Cheers
Jerry

I agree that mocking responses adds a potential layer of code rot if the API's change or some part of it changes e.g. requires API key, but then again the same could happen with the API tests in the sense that if the external API changes the tests don't automatically run (because they only run on code changes in this repo).

I think it might make sense to have both tests, if for nothing else it makes it easier for contributors to submit PRs and have test verification run (because travis-ci can't run PR builds using the encrypted API keys because it'd be a security risk because a PR could echo $GOOGLE_API_KEY and expose a key).

I like to think that the project could have 2 types of tests:

• unit tests - which test internal functionality and test against interfaces and mocks for external dependencies
• integration tests - actual test external API's in a full blown way

It could be that we "record" the results of the integration test runs as input for future unit tests, and we could have a process to sync that so we could keep it up to date.

I might explore those options and see how it looks and we can figure out if it'd make sense to add it (I definitely agree there's a trade off of adding the complexity/maintenance of mock HTTP testing vs just verifying against the real HTTP API).

I love the idea of connecting the 2 types of tests, by (regularly) sourcing the real life responses from integration tests to unit tests.
The http_geocoder is pretty stable, the only heavy-lifting of geocoder for individual service provider is response parsing. Mocking HTTP responses makes sense to me now.

https://godoc.org/github.com/orchestrate-io/dvr sounds like an interesting way to handle HTTP testing, basically you can run it in record mode once (or when API changes occur or new endpoints added) and then run tests in playback mode.

Record HTTP Test Responses

go test -dvr.record .

Playback HTTP Test Responses

go test -dvr.replay .

Hi @dougnukem,
I recorded responses from Google Maps API and use a test http server to accept requests and respond corresponding responses. It's kind of like a halfway through solution.

8ca5463

Tests will call real server if optional parameter is not available.

Cool @codingsince1985 that looks like a great start, we can start with that (I think it'll be really helpful if there are community contributions so that the travis-ci tests can at least run against some example responses).

I was thinking we could toggle using the test HTTP data based on if there is an API key e.g. GOOGLE_API_KEY available as an environment variables (so in travis-ci pull-requests from outside the repo which can't use the encrypted variables it'd fallback to test data).

I made an attempt to use the https://godoc.org/github.com/orchestrate-io/dvr library which will record requests automatically, but playing them back got weird because it expected exact requests (which included API keys).

I do like the flow of the dvr library where you run the unit tests with a flag and it'll record the test HTTP requests and responses, it just needs a little more configurability on the playback.

#27

dvr does look promising, though

closed via #27