Login with: Google Twitter Facebook Local

Avoiding The Needless Multiplication Of Forms

Using OAuth2

Dr Andrew Moss


Finding weird problems

A while ago I decided that I would add some simple functionality for comments to this blog. Handling user identities is awkward and requires a lot of manual intervention, so allowing somebody else to manage user identities saves a lot of work. It also means that visitors don't need to sign up for yet-another-login just to leave comments on a site, as they can use an identity that they already have (e.g. Google account, Facebook etc).
The technology to allow this is called OAuth2 and it is quite simple to use. However when I wrote the code to use it six months ago I ran into a weird error, got distracted by resigning my last job and working out the notice period, and only got back to diagnosing it yesterday. There are no comments yet (going to start writing the code for them tomorrow, probably...) but today I want to document how the authentication works, and the process for finding the weird bug.

OAuth2 in Go

OAuth works via access tokens. When a user hits a page that you want to authenticate them on, they are pushed out to an external site (e.g. the Google Accounts page) to perform authentication. If they successfully login on the external page we receive a token that represents them. We can use the token to access an API on the external site and request information such as their display name.
http.Handle("/secure/", wrapper(http.HandlerFunc(secureHandler))) ... func secureHandler(out http.ResponseWriter, req *http.Request) { token,err := req.Cookie("login") if err==http.ErrNoCookie { target := fmt.Sprintf("../login.html?from=%s",req.URL.Path[1:]) // Lead leading slash // The golang Redirect is based off an obsolete RFC so it forces the URL to absolute out.Header().Set("Location",target) out.WriteHeader(http.StatusFound) return } else if err!=nil { http.Error(out, errors.New("Something went wrong :(").Error(), http.StatusInternalServerError) return } loginKey,ok := checkMac(token.Value) s,found := sessions[string(loginKey)] if !ok || !found { fmt.Printf("Bad session cookie: %s -> %s (%s,%s)\n", token.Value, loginKey, ok, found) target := fmt.Sprintf("../login.html?from=%s",req.URL.Path[1:]) // Drop leading slash out.Header().Set("Location",target) out.WriteHeader(http.StatusFound) return } else { fmt.Fprintf(out, "In you are, %s from %s\n",s.user.Name,s.provider) } }
This code operates in two modes:
The wrapper can be ignored - it just traps exceptions and does some logging within the handlers.
If we bounced off to the login page then choosing a provider sends us to the auth page with a parameter to record the provider:
var providers = map[string]*oauth2.Config{ "google": &oauth2.Config{ ClientID: "", // Loaded in from a file, not under source control ClientSecret: "", // Loaded in from a file, not under source control Endpoint: oauth2.Endpoint{ AuthURL: "https://accounts.google.com/o/oauth2/auth", TokenURL: "https://accounts.google.com/o/oauth2/token"}, RedirectURL: "https://mechani.se/awmblog/callback", Scopes: []string{"openid", "profile", "email" }}, "twitter": &oauth2.Config{ ... }, "facebook": ... } func authHandler(out http.ResponseWriter, req *http.Request) { provName := req.URL.Query().Get("provider") config,found := providers[provName] if !found { http.Error(out, "The mystery deepens!?", http.StatusInternalServerError) return } referer := req.Header.Get("Referer") refUrl,err := url.Parse(referer) if len(referer)==0 || err!=nil { http.Error(out, "Referer was made of hairy bollocks", http.StatusInternalServerError) return } original := refUrl.Query().Get("from") stateData := fmt.Sprintf("%s|%s",provName,original) encState := msgMac(stateData) http.Redirect(out, req, config.AuthCodeURL(encState), http.StatusFound) }
The point of this code is to take the selected provider and load their config data to decide how to redirect the user off to their authentication page. This will pop up the familiar "Choose your account" page, and if it is the first time signing in will show what data we want access to so that they can consent.
Note that we don't trust the parameters that come in from our own page (to guard against spoofing). Reparsing the embedded URL to validate it may not be be strong enough, I will probably go through the parsing code carefully to check what kind of thing might trip it up. We also guard the data they we are handing over with a MAC (again to guard against spoofing) and we need to verify it when the user comes back from the provider page.
func callbackHandler( out http.ResponseWriter, req *http.Request) { var provider, original string ctx := context.Background() state,ok := checkMac( req.URL.Query().Get("state") ) if ok { decodeParts := bytes.Split(state,[]byte("|")) provider = string(decodeParts[0]) original = string(decodeParts[1]) } else { http.Error(out, "State is always the problem", http.StatusBadRequest) return } config := providers[provider] oauth2Token,err := config.Exchange(ctx, req.URL.Query().Get("code")) if err!=nil { http.Error(out, fmt.Sprintf("Provider cannot exchange code: %s",err.Error()), http.StatusBadRequest) return } tokenSrc := oauth2.StaticTokenSource(oauth2Token) token,err := tokenSrc.Token() if err!=nil { http.Error(out, "Token problem", http.StatusInternalServerError) return } uiReq,err := http.NewRequest("GET", userInfos[provider],nil) if err!=nil { http.Error(out, "No request for userinfo", http.StatusInternalServerError) return } token.SetAuthHeader(uiReq) resp, err := doRequest(ctx, uiReq) if err!=nil { http.Error(out, fmt.Sprintf("Token request failed: %s",err.Error()), http.StatusInternalServerError) return } defer resp.Body.Close() body, err := ioutil.ReadAll(resp.Body) if err!=nil { http.Error(out, "Can't read Token request body", http.StatusInternalServerError) return } if resp.StatusCode != http.StatusOK { http.Error(out, fmt.Sprintf("Token exchange failed %s: %s", resp.Status, body), http.StatusInternalServerError) } var userInfo UserInfo if err := json.Unmarshal(body, &userInfo); err != nil { http.Error(out, fmt.Sprintf("Userinfo decode failed: %v", err), http.StatusInternalServerError) return } loginKey := fmt.Sprintf("%s|%s", provider, userInfo.Sub) encLogin := msgMac(loginKey) http.SetCookie(out, &http.Cookie{Name:"login", Value:encLogin, Expires:time.Now().Add(time.Minute*10)}) sessions[loginKey] = Session{user:userInfo,token:oauth2Token,provider:provider} http.Redirect(out, req, original, http.StatusFound) }
This code performs what is called "three-legged-authentication": after the user has interacted with the provider in the browser to login, the server needs to trade the opaque token received for useful details: name, email etc. The token really just guards access to an API that can be used to look stuff up.

Testing, bug finding

So when I wrote this originally, I setup the application details on Google for a test site: something running on localhost and checked that it all worked. Everything was fine and I landed on the logged in page with the userdata returned. Cool.
Then I moved it onto the live site, updated the app details at Google for the target domain... and it all broke horribly. The call to config.Exchange()started to fail with a horribly opaque error message: "x509: certificate signed by unknown authority". This was strange as it all worked fine on the test system, so how could the same code break on the live system?
First step in finding the problem is walking through each piece of the code shown above and checking that all of the values make sense. Everything that I could see seemed to be correct, although several of the values are opaque so there is no way to verfiy them. This seemed to be a dead-end.
Next step is to try to find out more about the error message, but googling it reveals nothing apart from an unrelated problem with a bad certificate on a google server two years ago. Another dead-end. This is the point at which the frustration level peaks...
Cue much circling through the Google App Developer Console and server logs not solving the problem at all.

Diagnostic process

So, when you are faced with a problem that seems to be unique (no help from stackoverflow or random google results) the only thing to do is to sit down and trace through the code step by step until you understand what the problem actually is. In the case of a modern API where most of the code hidden several levels of abstraction lower this will generally mean reading the source rather than trying to instrument it to see what it does.
So how does config.Exchange()actually verify the certificate is valid? First step is to find the code for the library and check what Exchange does... so we bounce through retrieveTokenand then internal.RetrieveTokenand then... no certifcate check. Huh?
So it looks like the problem is not in the piece of code that reports it and there are no clues for what is causing it. Hmmm, so we need to look somewhere else...
At this point we are looking for an unknown problem in an unknown piece of code. Ah, the fun kind of debugging! So we need some background knowledge, a flash of insight or a completely different kind of random googling...
Dropping the specific parts of the error message lets us look around a more general area, googling for "golang x509 freebsd" gives a discussion of a problem in an older Go runtime and the search paths for CA roots. Ah, so now the piece of background knowledge that we need is that the URL for the token exchange uses the https:// schema, so it needs to do an SSL handshake and because the three-legged-authentication means we are POSTing to accounts.google.com we need to verify a trust path for its certificate from the root. So those CA roots are stored in the system in a config file...
Which explains why the same code seems to work on the testing system, where it just runs as a binary in the host system and the production system where the web-server runs inside a service jail that only has a whitelisted list of files in the file-system!


The hard part is always figuring out the problem, the fix is trivial:
sudo cp /etc/ssl/cert.pem /usr/sj/jails/sj_awmblog/etc/ssl/
So, the fix itself is not that interesting, but hopefully the OAuth2 code is a good example for somebody to work from, and the diagnostic process might trigger a flash of insight for somebody else.


Andrew Moss/google at 07:46 commented:

Testing the comment system on the live server, this is my local (non-oauth) account.
print "Fingers crossed..."

amoss/local at 07:48 commented:

Haha, no that was my OAuth to Google. That is what happens when you stop testing one thing to fix a bug that pops up. This, however, is definitely my local account...

Andrew Moss/google at 07:52 commented:

  • Ooo!
  • It works from an ipad as well
  • Nice to know...

Sign in at the top of the page to leave a comment