Avoiding The Needless Multiplication Of Forms
Using OAuth2
Dr Andrew Moss2019-08-08
Finding weird problems
A while ago I decided that I would add some simple functionality for comments to this blog. Handling user identities is awkward and requires a lot of manual intervention, so allowing somebody else to manage user identities saves a lot of work. It also means that visitors don't need to sign up for yet-another-login just to leave comments on a site, as they can use an identity that they already have (e.g. Google account, Facebook etc).
The technology to allow this is called OAuth2 and it is quite simple to use. However when I wrote the code to use it six months ago I ran into a weird error, got distracted by resigning my last job and working out the notice period, and only got back to diagnosing it yesterday. There are no comments yet (going to start writing the code for them tomorrow, probably...) but today I want to document how the authentication works, and the process for finding the weird bug.
OAuth works via access tokens. When a user hits a page that you want to authenticate them on, they are pushed out to an external site (e.g. the Google Accounts page) to perform authentication. If they successfully login on the external page we receive a token that represents them. We can use the token to access an API on the external site and request information such as their display name.
http.Handle("/secure/", wrapper(http.HandlerFunc(secureHandler)))
...
func secureHandler(out http.ResponseWriter, req *http.Request) {
token,err := req.Cookie("login")
if err==http.ErrNoCookie {
target := fmt.Sprintf("../login.html?from=%s",req.URL.Path[1:]) // Lead leading slash
// The golang Redirect is based off an obsolete RFC so it forces the URL to absolute
out.Header().Set("Location",target)
out.WriteHeader(http.StatusFound)
return
} else if err!=nil {
http.Error(out, errors.New("Something went wrong :(").Error(),
http.StatusInternalServerError)
return
}
loginKey,ok := checkMac(token.Value)
s,found := sessions[string(loginKey)]
if !ok || !found {
fmt.Printf("Bad session cookie: %s -> %s (%s,%s)\n", token.Value, loginKey, ok, found)
target := fmt.Sprintf("../login.html?from=%s",req.URL.Path[1:]) // Drop leading slash
out.Header().Set("Location",target)
out.WriteHeader(http.StatusFound)
return
} else {
fmt.Fprintf(out, "In you are, %s from %s\n",s.user.Name,s.provider)
}
}
This code operates in two modes:
- A user has been logged in by OAuth2 authentication on a provider site. We recorded this by storing a login cookie with some session information (a MAC of their OAuth2 token and a record of which provider they used).
- A user that has not been logged in, redirect them to a page to choose a provider. We pass this landing page as a parameter so that at the end of the login process we can redirect back here.
The wrapper can be ignored - it just traps exceptions and does some logging within the handlers.
If we bounced off to the login page then choosing a provider sends us to the auth page with a parameter to record the provider:
var providers = map[string]*oauth2.Config{
"google": &oauth2.Config{
ClientID: "", // Loaded in from a file, not under source control
ClientSecret: "", // Loaded in from a file, not under source control
Endpoint: oauth2.Endpoint{
AuthURL: "https://accounts.google.com/o/oauth2/auth",
TokenURL: "https://accounts.google.com/o/oauth2/token"},
RedirectURL: "https://mechani.se/awmblog/callback",
Scopes: []string{"openid", "profile", "email" }},
"twitter": &oauth2.Config{
... },
"facebook": ...
}
func authHandler(out http.ResponseWriter, req *http.Request) {
provName := req.URL.Query().Get("provider")
config,found := providers[provName]
if !found {
http.Error(out, "The mystery deepens!?", http.StatusInternalServerError)
return
}
referer := req.Header.Get("Referer")
refUrl,err := url.Parse(referer)
if len(referer)==0 || err!=nil {
http.Error(out, "Referer was made of hairy bollocks", http.StatusInternalServerError)
return
}
original := refUrl.Query().Get("from")
stateData := fmt.Sprintf("%s|%s",provName,original)
encState := msgMac(stateData)
http.Redirect(out, req, config.AuthCodeURL(encState), http.StatusFound)
}
The point of this code is to take the selected provider and load their config data to decide how to redirect the user off to their authentication page. This will pop up the familiar "Choose your account" page, and if it is the first time signing in will show what data we want access to so that they can consent.
Note that we don't trust the parameters that come in from our own page (to guard against spoofing). Reparsing the embedded URL to validate it may not be be strong enough, I will probably go through the parsing code carefully to check what kind of thing might trip it up. We also guard the data they we are handing over with a MAC (again to guard against spoofing) and we need to verify it when the user comes back from the provider page.
func callbackHandler( out http.ResponseWriter, req *http.Request) {
var provider, original string
ctx := context.Background()
state,ok := checkMac( req.URL.Query().Get("state") )
if ok {
decodeParts := bytes.Split(state,[]byte("|"))
provider = string(decodeParts[0])
original = string(decodeParts[1])
} else {
http.Error(out, "State is always the problem", http.StatusBadRequest)
return
}
config := providers[provider]
oauth2Token,err := config.Exchange(ctx, req.URL.Query().Get("code"))
if err!=nil {
http.Error(out, fmt.Sprintf("Provider cannot exchange code: %s",err.Error()),
http.StatusBadRequest)
return
}
tokenSrc := oauth2.StaticTokenSource(oauth2Token)
token,err := tokenSrc.Token()
if err!=nil {
http.Error(out, "Token problem", http.StatusInternalServerError)
return
}
uiReq,err := http.NewRequest("GET", userInfos[provider],nil)
if err!=nil {
http.Error(out, "No request for userinfo", http.StatusInternalServerError)
return
}
token.SetAuthHeader(uiReq)
resp, err := doRequest(ctx, uiReq)
if err!=nil {
http.Error(out, fmt.Sprintf("Token request failed: %s",err.Error()),
http.StatusInternalServerError)
return
}
defer resp.Body.Close()
body, err := ioutil.ReadAll(resp.Body)
if err!=nil {
http.Error(out, "Can't read Token request body", http.StatusInternalServerError)
return
}
if resp.StatusCode != http.StatusOK {
http.Error(out, fmt.Sprintf("Token exchange failed %s: %s", resp.Status, body),
http.StatusInternalServerError)
}
var userInfo UserInfo
if err := json.Unmarshal(body, &userInfo); err != nil {
http.Error(out, fmt.Sprintf("Userinfo decode failed: %v", err),
http.StatusInternalServerError)
return
}
loginKey := fmt.Sprintf("%s|%s", provider, userInfo.Sub)
encLogin := msgMac(loginKey)
http.SetCookie(out, &http.Cookie{Name:"login",
Value:encLogin,
Expires:time.Now().Add(time.Minute*10)})
sessions[loginKey] = Session{user:userInfo,token:oauth2Token,provider:provider}
http.Redirect(out, req, original, http.StatusFound)
}
This code performs what is called "three-legged-authentication": after the user has interacted with the provider in the browser to login, the server needs to trade the opaque token received for useful details: name, email etc. The token really just guards access to an API that can be used to look stuff up.
So when I wrote this originally, I setup the application details on Google for a test site: something running on localhost and checked that it all worked. Everything was fine and I landed on the logged in page with the userdata returned. Cool.
Then I moved it onto the live site, updated the app details at Google for the target domain... and it all broke horribly. The call to config.Exchange()started to fail with a horribly opaque error message: "x509: certificate signed by unknown authority". This was strange as it all worked fine on the test system, so how could the same code break on the live system?
First step in finding the problem is walking through each piece of the code shown above and checking that all of the values make sense. Everything that I could see seemed to be correct, although several of the values are opaque so there is no way to verfiy them. This seemed to be a dead-end.
Next step is to try to find out more about the error message, but googling it reveals nothing apart from an unrelated problem with a bad certificate on a google server two years ago. Another dead-end. This is the point at which the frustration level peaks...
Cue much circling through the Google App Developer Console and server logs not solving the problem at all.
So, when you are faced with a problem that seems to be unique (no help from stackoverflow or random google results) the only thing to do is to sit down and trace through the code step by step until you understand what the problem actually is. In the case of a modern API where most of the code hidden several levels of abstraction lower this will generally mean reading the source rather than trying to instrument it to see what it does.
So how does
config.Exchange()actually verify the certificate is valid? First step is to find the
code for the library and check what Exchange does... so we bounce through
retrieveTokenand then
internal.RetrieveTokenand then... no certifcate check. Huh?
So it looks like the problem is not in the piece of code that reports it and there are no clues for what is causing it. Hmmm, so we need to look somewhere else...
At this point we are looking for an unknown problem in an unknown piece of code. Ah, the fun kind of debugging! So we need some background knowledge, a flash of insight or a completely different kind of random googling...
Dropping the specific parts of the error message lets us look around a more general area, googling for "golang x509 freebsd" gives a discussion of a problem in an older Go runtime and the search paths for CA roots. Ah, so now the piece of background knowledge that we need is that the URL for the token exchange uses the https:// schema, so it needs to do an SSL handshake and because the three-legged-authentication means we are POSTing to accounts.google.com we need to verify a trust path for its certificate from the root. So those CA roots are stored in the system in a config file...
Which explains why the same code seems to work on the testing system, where it just runs as a binary in the host system and the production system where the web-server runs inside a service jail that only has a whitelisted list of files in the file-system!
The hard part is always figuring out the problem, the fix is trivial:
sudo cp /etc/ssl/cert.pem /usr/sj/jails/sj_awmblog/etc/ssl/
So, the fix itself is not that interesting, but hopefully the OAuth2 code is a good example for somebody to work from, and the diagnostic process might trigger a flash of insight for somebody else.
Comments
Andrew Moss/google at 07:46 commented: