In my final year of college, I got swept up in creating Twitter bots and tools. As a result, I created several different projects in Go but the go-twitter package hadn’t been updated in over a year and was missing the majority of API functionality. At first, I submitted pull requests to the original library but received sparse answers and a never-ending list of semantic changes (despite passing the Continuous Integration tests). Looking back, there was only one maintainer and he had a life and certain standards but in the moment, I was pretty frustrated. I ended up forking the repository and working on making it API-complete. I doubled the API coverage within a week so a large learning curve for me was how do I write unit-tests for the Twitter API requests? Without needing to actually interact with the API?

The answer was surprisingly easier than I expected: reroute requests to a local server and mock the API endpoints with their return data.

Firstly, we’ve got some imports to handle. I’m going to import some testing libraries as part of this because we’re going to write some custom tests later on to finish the unit-testing portion of running the local server.


import (
    "net/http"
    "net/http/httptest"
    "net/url"
    "testing"

    "github.com/stretchr/testify/assert"
)

A small hiccup I didn’t consider at first was that the Twitter API operates via HTTPS and, obviously, my tiny little test server wasn’t going to be able to use Twitter’s SSL Cert for communication. To get around this, we need to rewrite the transport protocol from HTTPS to HTTP.


// RewriteTransport rewrites https requests to http to avoid TLS cert issues
// during testing.
type RewriteTransport struct {
    Transport http.RoundTripper
}

// RoundTrip rewrites the request scheme to http and calls through to the
// composed RoundTripper or if it is nil, to the http.DefaultTransport.
func (t *RewriteTransport) RoundTrip(req *http.Request) (*http.Response, error) {
    req.URL.Scheme = "http"
    if t.Transport == nil {
        return http.DefaultTransport.RoundTrip(req)
    }
    return t.Transport.RoundTrip(req)
}

Next, we’re going to write a small server that proxies all requests to our local server rather than allowing it to communicate with the outside world.


// testServer returns an http Client, ServeMux, and Server. The client proxies
// requests to the server and handlers can be registered on the mux to handle
// requests. The caller must close the test server.
func testServer() (*http.Client, *http.ServeMux, *httptest.Server) {
    mux := http.NewServeMux()
    server := httptest.NewServer(mux)
    transport := &RewriteTransport{&http.Transport{
        Proxy: func(req *http.Request) (*url.URL, error) {
            return url.Parse(server.URL)
        },
    }}
    client := &http.Client{Transport: transport}
    return client, mux, server
}

In the above, we’re returning:

  • A HTTP Client: allows us to make calls to the API and handles the proxy to our local server
  • A HTTP request multiplexer: It matches the URL of each incoming request against a list of registered patterns and calls the handler for the pattern that most closely matches the URL.
  • A HTTP Server: To process and parse the requests from the multiplexer.

To finish off our test server, it’d be useful to have some HTTP-specific assertion functions for our unit-testing. The following functions are for: asserting the HTTP methods match, asserting the query parameters match and asserting that the data in a form POST match the proper key-value pairs.


// assertMethod tests that the incoming Request method matches the expected method
func assertMethod(t *testing.T, expectedMethod string, req *http.Request) {
    assert.Equal(t, expectedMethod, req.Method)
}

// assertQuery tests that the Request has the expected url query key/val pairs
func assertQuery(t *testing.T, expected map[string]string, req *http.Request) {
    queryValues := req.URL.Query()
    expectedValues := url.Values{}
    for key, value := range expected {
        expectedValues.Add(key, value)
    }
    assert.Equal(t, expectedValues, queryValues)
}

// assertPostForm tests that the Request has the expected key values pairs url
// encoded in its Body
func assertPostForm(t *testing.T, expected map[string]string, req *http.Request) {
    req.ParseForm() // parses request Body to put url.Values in r.Form/r.PostForm
    expectedValues := url.Values{}
    for key, value := range expected {
        expectedValues.Add(key, value)
    }
    assert.Equal(t, expectedValues, req.Form)
}

Usage

The above wouldn’t be of any use to you unless I provided an example of how I used it. Below is some code from my go-twitter library that tests the ability to mute/unmute a person a user follows.


import (
    "fmt"
    "net/http"
    "testing"

    "github.com/stretchr/testify/assert"
)

func TestFriendship_Update(t *testing.T) {
    httpClient, mux, server := testServer()
    defer server.Close()

    mux.HandleFunc("/1.1/friendships/update.json", func(w http.ResponseWriter, r *http.Request) {

        // Assert that the HTTP methods match
        assertMethod(t, "POST", r)

        // Check if the query sent is the query expected
        assertQuery(t, map[string]string{"screen_name": "thejokersthief", "retweets": "true"}, r)

        // Declare you're returning JSON
        w.Header().Set("Content-Type", "application/json")
        
        // Print out example API output
        fmt.Fprintf(w, `{"relationship":{"source":{"id":54655541,"id_str":"54655541","screen_name":"TheJokersThief","following":false,"followed_by":false,"live_following":false,"following_received":false,"following_requested":false,"notifications_enabled":false,"can_dm":false,"blocking":false,"blocked_by":false,"muting":false,"want_retweets":true,"all_replies":false,"marked_spam":false},"target":{"id":623265148,"id_str":"623265148","screen_name":"dghubble","following":false,"followed_by":false,"following_received":false,"following_requested":false}}}`)
    })

    // Define expected values from the API (based on the JSON we print out above)
    expected := &FriendshipShowResult{
        Relationship: FriendshipRelationship{
            Target: FriendshipRelationshipTarget{
                IDStr:      "623265148",
                ID:         623265148,
                ScreenName: "dghubble",
                Following:  false,
                FollowedBy: false,
            },
            Source: FriendshipRelationshipSource{
                CanDM:                false,
                Blocking:             false,
                Muting:               false,
                IDStr:                "54655541",
                AllReplies:           false,
                WantRetweets:         true,
                ID:                   54655541,
                MarkedSpam:           false,
                ScreenName:           "TheJokersThief",
                Following:            false,
                FollowedBy:           false,
                NotificationsEnabled: false,
            },
        },
    }
    
    client := NewClient(httpClient)
    params := &FriendshipUpdateParams{
        ScreenName: "thejokersthief",
        Retweets:   true,
    }
    friendshipStatus, _, err := client.Friendships.Update(params)
    assert.Nil(t, err)
    assert.Equal(t, expected, friendshipStatus)
}

Leave a Reply