The other day, I was reading a website about some historical documents, [when I saw][tweet] that it had an error message on top: {{
}} [tweet]: https://twitter.com/carlmjohnson/status/1286302974780809217 Some quick searching online for the error message revealed that it was caused by a mismatch between the site’s versions of PHP and WordPress. Older versions of WordPress had a bug in the `switch` statement of a certain localization component, and later versions of PHP dump a warning about this bug out to the end user HTML. When I came back to the site a few days later, it had been fixed. The ultimate reason for my seeing for the error message as a random reader is that PHP has [too many ways to deal with errors](https://lukeplant.me.uk/blog/posts/wordpress-4.7.2-post-mortem/): > Builtin PHP functions, and therefore any PHP project, have a whole range of error handling mechanisms — errors, warnings, returning error values, and exceptions. At every point, calling code needs to know which system will be used to handle errors. PHP often chooses to send warnings like this right out to the end user because it doesn’t trust operators to actually read their Apache logs. Such a practice would be very out of place in the Go programming language. - - - - In Go, errors are values. While it’s true that Go has both errors and panics, so that Go is theoretically the same as Java with [checked][c2c] and [unchecked][c2u] exceptions, there are also important differences. Because errors are normal values instead of a special form of control flow, they have the same flexibility ([and inflexibility][nil]) as other values. As [Rob Pike explains][eav], [c2c]: https://wiki.c2.com/?CheckedException [c2u]: https://wiki.c2.com/?UncheckedException [nil]: https://stackoverflow.com/questions/18771569/avoid-checking-if-error-is-nil-repetition > Values can be programmed, and since errors are values, errors can be programmed. Second, because the `error` interface is pervasively used instead of concrete error types, it is possible to change underlying error implementations without having to change the types returned all the way up a call chain. With checked exceptions, if today my function can only throw a `FileException`, then tomorrow I cannot start throwing a `URLException` without breaking any callers depending on my function having only one possible exception type. (The lead architect of C# [cited this problem][csharp] as one of his reasons for not adding checked exception to that language.) In Go, the use of the simple `error` interface everywhere prevents callers from being too dependent on the exact type of errors that a function returns. [csharp]: https://www.artima.com/intv/handcuffs.html#part2 Go is a statically typed language, but the pervasive use of the `error` interface allows for runtime dynamic type introspection. The dynamic nature of errors can [lead to problems][cockroach] if misused, but overall, has allowed a number of community experiments in [annotating errors][dave], culminating in the inclusion of the [`errors.As` function][eaprop] in the [Go 1.13 standard library][ew113] in [September 2019][go113]. The [docs for errors.As][eadoc] explain: [cockroach]: https://github.com/cockroachdb/cockroach/blob/master/docs/RFCS/20190318_error_handling.md [dave]: https://dave.cheney.net/2016/04/27/dont-just-check-errors-handle-them-gracefully [eaprop]: https://go.googlesource.com/proposal/+/master/design/29934-error-values.md [ew113]: https://golang.org/doc/go1.13#error_wrapping [eav]: https://blog.golang.org/errors-are-values [go113]: https://golang.org/doc/devel/release.html#go1.13 [eadoc]: https://pkg.go.dev/errors?tab=doc#As > `func As(err error, target interface{}) bool` > > `As` finds the first error in `err`'s chain that matches target, and if so, sets `target` to that error value and returns true. Otherwise, it returns false. > > The chain consists of `err` itself followed by the sequence of errors obtained by repeatedly calling `Unwrap`. And it provides an example: ```go if _, err := os.Open("non-existing"); err != nil { var pathError *os.PathError if errors.As(err, &pathError) { fmt.Println("Failed at path:", pathError.Path) } else { fmt.Println(err) } } ``` So while a particular value may be statically typed as `error`, it may also contain a more useful type dynamically available in its `Unwrap` chain for consumers to programmatically introspect. This gives Go the ability to create [inheritance trees for errors][inherit] without the baggage of an actual classical object inheritance system. [inherit]: https://twitter.com/davidcrawshaw/status/1238081069540311042 I have created two libraries for working errors that use `errors.As` since it was announced last year, called [exitcode][ec] and [resperr][re]. It may be useful for me to explain the philosophy behind them and how to use them here, since I think they could inspire similar projects in other domains. [ec]: https://github.com/earthboundkid/exitcode [re]: https://github.com/earthboundkid/resperr - - - - First, let me explain [package exitcode][pexc]. When you run a process in a Unix-like system, it has an "exit code". Zero indicates that the program ran successfully, and any other code indicates a failure. There have been [various][xcode] [attempts][bsd] to standardize general purpose exit codes, but none have stuck. Most programs either use only 0 and 1 or they have custom set of codes. For example, [curl](https://curl.haxx.se/libcurl/c/libcurl-errors.html) defines 25 as "upload failed" and 47 as "too many redirects" and so forth on up to 96 "QUIC connection error". [xcode]: https://www.tldp.org/LDP/abs/html/exitcodes.html [bsd]: https://www.freebsd.org/cgi/man.cgi?query=sysexits&sektion=3 [pexc]: https://pkg.go.dev/github.com/carlmjohnson/exitcode?tab=doc The exitcode package is a simple library to help you [write a CLI in Go][cli] that returns a proper exit code. Of course, the simplest helper would just be function that returns 0 if error is nil and 1 if it is non-nil, but we can do more than that, thanks to `errors.As`. [cli]: /post/2020/go-cli-how-to-and-advice/ Package exitcode documents a `Coder` interface extension to `error`: ```go type Coder interface { error ExitCode() int } ``` This lets you define an error type and provide a custom exit code to associate with your error. `exitcode.Get` is defined to return 0 for `nil`, return 1 for unknown errors, and use `errors.As` to search through the `Unwrap` chain of errors for anything defining a `Coder`. If it finds one, it returns that custom return value. To make it more convenient, package exitcode also has a helper function called `exitcode.Set(error, int) error` which wraps an error in an unexported `Coder` implementation, so that you can easily set a custom exit code without having to define your own custom error type. So, for example, if you were rewriting curl in Go, you might create an `http.Client` with a `CheckRedirect` policy that returns `exitcode.Set(err, 47)` if it sees that a request has been redirected too many times. Other error handlers in the chain between the redirect checker and the [one line main function][cli] can just pass the error along without being aware that it has a custom exit code associated with it. At the top level the CLI can bottom out in a call to `exitcode.Exit(error)`, which is a convenience function for `os.Exit(exitcode.Get(error))`. - - - - Package exitcode is a simple example of what is possible by treating errors as dynamically typed values, but [package resperr][presp] takes it further. To understand the thinking behind resperr, I first need to talk about a blog post called [Failure is your Domain][ben] by Ben Johnson (no relation). The post builds on Rob Pike's [Error handling in Upspin][up] by documenting a philosophy for dealing with errors. Johnson writes that [ben]: https://middlemost.com/failure-is-your-domain/ [up]: https://commandcenter.blogspot.com/2017/12/error-handling-in-upspin.html > The tricky part about errors is that they need to be different things to different consumers of them. In any given system, we have at least 3 consumer roles—_the application, the end user, & the operator._ The article is worth reading in full to think about how these roles interact, but suffice it to say, towards that goal, Johnson defines a `struct` containing different fields meant for the application, end user, and operator: ```go // Error defines a standard application error. type Error struct { // Machine-readable error code. Code string // Human-readable message. Message string // Logical operation and nested error. Op string Err error } ``` Core to Johnson's proposal is a set of application error codes, which he argues ought to be worked out for the specific domain of an application. In his case, they look like this: ```go // Application error codes. const ( ECONFLICT = "conflict" // action cannot be performed EINTERNAL = "internal" // internal error EINVALID = "invalid" // validation failed ENOTFOUND = "not_found" // entity does not exist ) ``` Johnson's article predates `errors.As`, so in it he explains how to dig through an error chain manually to retrieve machine codes and user messages from `error` interface values, instead of relying on the existence of the `errors.As` mechanism. One last quote from the article: > Error handling is a critical piece of your application design and is complicated by the variety of different consumer roles that require error information. By considering error codes, error messages, and logical stack traces in our design we can fulfill the needs of each consumer. By integrating our `Error` into our domain, we give all parts of our application a common language to communicate about when unexpected things happen. Failure to think clearly about the separate roles of the application, the end user, and the operator in dealing with errors is exactly what led old PHP applications to dump [potentially dangerous][owasp] error messages about database failures or application bugs out to the final HTML to end users instead of logging them for operators. Those systems were built without thinking about the difference between the information a operator needs to debug an overloaded server and the information needed by a website reader (or a website attacker!). [owasp]: https://owasp.org/www-community/Improper_Error_Handling Johnson's article was very influential on my thinking as I was building a web application with a Go HTTP JSON backend. As I worked on it over a series of months, I realized two things: first that my failure domain just was the set of [HTTP status codes][status], and second that in a majority of cases (but not quite all), my user message was a restatement of the status code. I wrote [package resperr][presp] with these realizations in mind. [status]: https://en.wikipedia.org/wiki/List_of_HTTP_status_codes [presp]: https://pkg.go.dev/github.com/carlmjohnson/resperr?tab=doc - - - - Package resperr defines two interfaces to extend errors: one for HTTP status codes and another for user messages. ```go type StatusCoder interface { error StatusCode() int } type UserMessenger interface { error UserMessage() string } ``` This is similar to package exitcode with its `Coder` interface, but an important difference is the relationship between the two interfaces. The HTTP status codes have default user messages associated with them already, which are the "reason phrases" of [RFC 7231][rfc]. Go provides the [`http.StatusText(int) string`][stext] function to look up the status text from a status code. Putting these together, the docstring for `resperr.UserMessage(error) string` looks like this: [rfc]: https://tools.ietf.org/html/rfc7231#section-6.1 [stext]: https://pkg.go.dev/net/http?tab=doc#StatusText > `UserMessage` returns the user message associated with an error. If no message is found, it checks `StatusCode` and returns that message. Because the default status is 500, the default message is `"Internal Server Error"`. If `err` is `nil`, it returns `""`. Finally, let's look at a [short demonstration][demo] of how resperr could be used to write an HTTP JSON API. First, we need to write a short helper function to send errors to our logging system for capture while also returning them to end users: [demo]: https://github.com/carlmjohnson/resperr/blob/master/example_test.go ```go func replyError(w http.ResponseWriter, r *http.Request, err error) { logError(w, r, err) code := resperr.StatusCode(err) msg := resperr.UserMessage(err) replyJSON(w, r, code, struct { Status int `json:"status"` Message string `json:"message"` }{ code, msg, }) } ``` Then in our handler, we call the helper any time something goes wrong: ```go func myHandler(w http.ResponseWriter, r *http.Request) { // ... check user permissions... if err := hasPermissions(r); err != nil { replyError(w, r, err) return } // ...validate request... n, err := getItemNoFromRequest(r) if err != nil { replyError(w, r, err) return } // ...get the data ... item, err := getItemByNumber(n) if err != nil { replyError(w, r, err) return } replyJSON(w, r, http.StatusOK, item) } ``` In the functions that the handler is calling, we can set appropriate errors, like a 404 Not Found for item not found while falling back to 500 Internal Server Error for unexpected errors: ```go func getItemByNumber(n int) (item *Item, err error) { item, err := dbCall("...") if err == sql.ErrNoRows { // this is an anticipated 404 return nil, resperr.New( http.StatusNotFound, "%d not found", n) } if err != nil { // this is an unexpected 500! return nil, err } // ... } ``` Similarly, `hasPermissions` can return 403 Forbidden and `getItemNoFromRequest` can return 400 Bad Request as needed. But for 400 Bad Request, we may want a more extensive user message: ```go func getItemNoFromRequest(r *http.Request) (int, error) { ns := r.URL.Query().Get("n") if ns == "" { return 0, resperr.WithUserMessage( resperr.New( http.StatusBadRequest, "missing ?n= in query"), "Please enter a number.") } n, err := strconv.Atoi(ns) if err != nil { return 0, resperr.WithCodeAndMessage( err, http.StatusBadRequest, "Input is not a number.") } return n, nil } ``` In real code, `getItemNoFromRequest` would probably just be part of the handler unless multiple routes needed the same query handling. - - - - So that's how I made package exitcode and package resperr, but the great thing about this pattern is it's widely applicable. You could make your own package ([perhaps in an afternoon?][deps]) for [gRPC errors][grpc], [FTP errors][ftp], [STMP errors][stmp], [LDAP errors][ldap], [CORBA errors][corba], or even [SOAP errors][soap]. If your application has its own set of error conditions, you could make up custom error codes just for your application, as Upspin does and Ben Johnson recommends. [deps]: /post/2020/avoid-dependencies/ [grpc]: https://github.com/grpc/grpc-go/blob/v1.12.0/codes/codes.go [ftp]: https://en.wikipedia.org/wiki/List_of_FTP_server_return_codes [stmp]: https://en.wikipedia.org/wiki/List_of_SMTP_server_return_codes [ldap]: https://ldapwiki.com/wiki/LDAP%20Result%20Codes [corba]: https://www.ibm.com/support/knowledgecenter/en/SSB23S_1.1.0.14/com.ibm.java.lnx.80.doc/diag/appendixes/corba_minor_codes/corba_minor_codes.html [soap]: https://developer.nhs.uk/apis/dos-api/soap_api_errors.html#error-codes One thing that package resperr doesn't handle yet are redirects because I've just been using it for a JSON API. Someone using it for a traditional server side rendered HTML web application might want to add that functionality. The key is that `errors.As` makes it easy to create error systems that work for your particular applications, users, and operators without being straitjacketed by the language into a one-size-fits-all approach that inadvertently exposes users to the internal operations of your system. Don't let your end users be distracted by irrelevant warning messages. Handle errors properly by thinking about their roles and domain within your application.