Printing Go types as S-Expressions
If you have been around Go for a while, you likely know that you can get a reasonable output from go structs when you use the ‘fmt’ package to render to the console. Let’s imagine I have the following types defined.
type Inner struct {
InnerString string
}
type Struct struct {
StringField string
NumberField float64
InnerField Inner
}If I print an instance of these using fmt, I get the following,
fmt.Println(i)
{Hello 1.23 {World}}Not bad but if I then print these with fmt.Printf and the %s format string.
fmt.Printf("%s\n", i)
{Hello %!s(float64=1.23) {World}}Ok, that’s not what I was expecting, but if we use %v we are back to the orginal.
fmt.Printf("%v\n", i)
{Hello 1.23 {World}}This behaviour is all documented in the manual which additionally indicates I can use %+v, which will add my field names.
fmt.Printf("%+v\n", i)
{StringField:Hello NumberField:1.23 InnerField:{InnerString:World}}One item missing from the default methods, they do not tell me the type being printed. I can fix this issue by embedding the type in the print statement.
fmt.Println("Struct = ", i)I can now figure out the type which can help when I’m trying to debug an issue or when I am trying to figure out the flow of a program.
I don’t know about you but I don’t find these defaults that helpful and I end up defining custom String methods for many of my types.
One day I got a little frustrated and thought, what if I could generate a ‘lisp’ style S-Expression from my Go structures?
Wouldn’t that be easier to read? Imagine if my log traces looked more like the following,
( Struct "Hello" 1.23 ( Inner "World" ) )I don’t need the field names if these are in the same order as declared in the structure, but I do get to know which types are being printed. To my eye these are easier to read, especially if this is a long list coming from a file or database query.
So the question is, how do we do this? Well, I could create a function for my types that do this. I could implement the ‘stringer’ method and make this work ‘auto-magically.’
func (i Inner) String() string {
return fmt.Sprintf("( Inner \"%s\" )", i.InnerString)
}
func (s Struct) String() string {
return fmt.Sprintf("( Struct \"%s\" %f %s )", s.StringField, s.NumberField, s.InnerField)
}That wasn’t hard at all, and when I printed the result with fmt.Println I get what I expect.
( Struct "Hello" 1.230000 ( Inner "World" ) )Great, but there is an odd side effect here; if I define the stringer interface and print using %+v, what would you expect to get? Not this,
( Struct "Hello" 1.230000 ( Inner "World" ) )That’s a feature/bug in the fmt code that I will not be fixing but is worth knowing.
I can go through all my code and implement my custom printing routines, the world is great. Well, is it? What happens if I use the stringer interface to format the data for another use case? Perhaps to pass to a logger that needs a specific format? Ok, so I can define a new function on my type, let’s define MyString.
func (i Inner) MyString() string {
return fmt.Sprintf("( Inner \"%s\" )", i.InnerString)
}
func (s Struct) MyString() string {
return fmt.Sprintf("( Struct \"%s\" %f %s )", s.StringField, s.NumberField, s.InnerField)
}Ok, that should do it,
fmt.Println(i.MyString())
( Struct "Hello" 1.230000 {World} )Ah, not so fast; my Inner struct is calling the default String method and is no longer converting correctly. To fix this, I have to modify my functions to call the special MyString function.
func (s Struct) MyString() string {
return fmt.Sprintf("( Struct \"%s\" %f %s )", s.StringField, s.NumberField, s.InnerField.MyString())
}And now we are back to my desired result.
( Struct "Hello" 1.230000 ( Inner "World" ) )Not only is this tedious, but it is also error-prone. I will miss one of these calls and won’t get the result I am expecting. So why not implement this in the way the standard library prints objects? That’s the approach I’ll take.
First, I define a type alias for the parts of my expression and an interface. I also define a stringer implementation for the NodeList.
I define the ‘Lisper’ interface to override the behavior for any types in my system that do not look quite right, or if I want to avoid printing some sensitive data to a log file. Sometimes you don’t want to print everything out …
// NodeList a list of nodes convert to string
type NodeList []string
func (nl NodeList) String() string {
return strings.Join(nl, " ")
}
// Lisper when implemented returns a slice of nodes and overrides
// the default conversion to a lisp node this is akin to the
// behaviour of the stringer interface
type Lisper interface {
LispString() []string
}You’ll note that I quote’ my strings in the output to make the distinction between the types and strings. I can achieve the desired result with a simple
fmt.Sprint( “\””, value, “\”” )There is a problem here, what happens when my string contains a quote character? Ideally, it would be escaped, it looks very odd to have un-matched, un-escaped quotes, and good luck building a parser to read in these expressions. A RegEx and a routine to replace any found quotes or special characters with an escaped quote works for these simple cases.
var quote = regexp.MustCompile("[,\"]")
// quoted replaces all quotes with escaped quotes and
// wraps the string in quotes
func quoted(n string, always bool) string {
if quote.Match([]byte(n)) {
return "\"" + strings.ReplaceAll(n, "\"", "\\\"") + "\""
} else if always {
return "\"" + n + "\""
} else {
return n
}
}The next step is to make this more general, to achieve this, we need to use reflection. Reflection in Go is compelling but often overused. For this use case, it fits the bill perfectly.
The core of this method is the recursive application of a function; since we want to be able to modify this function in the future, say to limit stack depth … we make the core routine private but, wrap it with an exported function.
// Run convert a value to a lisp style string
func Run(v interface{}) NodeList {
return run(v)
}
// run is the workhorse function, it is called recursively and returns a stack of lisp nodes
func run(v interface{}) []string {
// Code goes here ...
}I highly recommend providing these little indirections to the inner workings of your code. The usage of this function is as follows
fmt.Println(lisper.Run(i))
( Struct "Hello" 1.23 ( Inner "World" ) )The refection based implementation
Firstly, we short circuit the recursion if the type has a definition for our ‘lisper’ interface. This escape hatch allows a user of the utility to define their own behaviors when needed.
// Check to see if the Lisper interface is implemented.
// If so return the custom node stack
if l, ok := v.(Lisper); ok {
return l.LispString()
}One of the most satisfying parts of Go is that such seemingly simple code solves a whole lot of complexity.
Next, we use reflection on the supplied value to get the runtime type and the value of the type.
val := reflect.Indirect(reflect.ValueOf(v))
typ := val.Type()The ‘Indirect’ here is super important as this allows the caller to pass either a value or a pointer to the function, and we can ignore the details. Next, we pass the type into a type switch by looking at the ‘Kind’ of the type. You’ll notice that there are only a tiny number of Kinds that we have to handle here, and for everything else, we define a default implementation that turns the ‘value’ into a string. Finally, you’ll notice that we specialize the string type as we want to ensure that the value is always quoted. Otherwise, we only quote values that meet our RegEx.
switch typ.Kind() {
case reflect.Interface, reflect.Struct:
{
// CODE
}
case reflect.Slice:
{
// CODE
}
case reflect.Map:
{
// CODE
}
case reflect.String:
{
node := fmt.Sprint(v)
return []string{quoted(node, true)}
}
default:
{
// Use the default string methods
node := fmt.Sprint(v)
return []string{quoted(node, false)}
}
}Dealing with slices is relatively easy,
case reflect.Slice:
{
res := []string{"["}
for i := 0; i < val.Len(); i++ {
elemValue := val.Index(i)
res = append(res, run(elemValue.Interface())...)
}
res = append(res, "]")
return res
}We first chose a starting character, in our case “[“, and loop across each slice element, recursively calling our ‘run’ method. Next, we ‘splat’ the call results using ellipsis into our return data structure and terminate with a closing bracket “]”.
Sidebar, we use a slice of strings (our NodeList) to build up the final string and then join them into a last string with the strings.Join function. This avoids us constantly reallocating and copying strings. Another approach would be to use strings.Builder.
We model Struts and Interfaces in essentialy the same way as a slice. However, we must be careful to not ask for the value of the un-exported field.
case reflect.Interface, reflect.Struct:
{
res := []string{"("}
res = append(res, typ.Name())
for i := 0; i < typ.NumField(); i++ {
if typ.Field(i).IsExported() {
fieldValue := val.Field(i)
res = append(res, run(fieldValue.Interface())...)
}
}
res = append(res, ")")
return res
}Lastly, we handle the Maps; the naive implementation for a map looks very similar to the Slice. One of the gotchas with go maps is that the order of the keys is not determined, which is in general a good thing. I chose to order the keys so that our data printing is always consistent and helps with tests.
case reflect.Map:
{
// Keys are in a random order from a map, we sort the output to ensure stability
type kv struct {
key []string
value []string
}
kvs := make([]kv, val.Len())
for i, key := range val.MapKeys() {
value := val.MapIndex(key)
kvs[i] = kv{key: run(key.Interface()), value: run(value.Interface())}
}
sort.Slice(kvs, func(i, j int) bool {
return compare(kvs[i].key, kvs[j].key)
})
res := []string{"{"}
for _, kv := range kvs {
res = append(res, "(")
res = append(res, kv.key...)
res = append(res, ":")
res = append(res, kv.value...)
res = append(res, ")")
}
res = append(res, "}")
return res
}If you look closely at the sort you’ll notice a custom compare function, this looks like
// compare two arrays of strings
func compare(l []string, r []string) bool {
for i, j := range l {
if i > len(r) {
return true
}
if j > r[i] {
return false
}
}
return true
}We do this since Go maps can have other types of keys that are not strings, including structs and fixed sized arrays. We have turned these keys into NodeLists and need to be able to compare these slices of strings.
That’s all there is to the utility; full runnable code is below, but first, here is a go example that acts as a test.
// lisper_test.go
package lisper_test
import (
"fmt"
"surface/lisper"
)
//https://go.dev/blog/examples
type Custom string
func (c Custom) LispString() []string {
return []string{fmt.Sprintf("Prefix:%s", c)}
}
func Example() {
// Create some simple scalar values
fmt.Println(lisper.Run("A String"))
fmt.Println(lisper.Run(6.45))
fmt.Println(lisper.Run("A \"quoted\" String"))
// Create a struct in a struct
type Inner struct {
InnerString string
}
type Struct struct {
StringField string
NumberField float64
InnerField Inner
}
// Dump the empty struct
fmt.Println(lisper.Run(Struct{}))
// Dump the struct with values
fmt.Println(lisper.Run(Struct{
StringField: "Hello",
NumberField: 1.23,
InnerField: Inner{
InnerString: "World",
},
}))
// Custom handler
fmt.Println(lisper.Run(Custom("Suffix")))
// A list of strings
fmt.Println(lisper.Run([]string{"hello", "world"}))
// A map of float to string
fmt.Println(lisper.Run(map[string]float64{"a": 1.23, "x": 7.68, "b": 4.56}))
// Output:
//"A String"
//6.45
//"A \"quoted\" String"
//( Struct "" 0 ( Inner "" ) )
//( Struct "Hello" 1.23 ( Inner "World" ) )
//Prefix:Suffix
//[ "hello" "world" ]
//{ ( "a" : 1.23 ) ( "b" : 4.56 ) ( "x" : 7.68 ) }
}I find using examples with tests a great way of documenting utilities such as this one. It’s an often over looked feature of the built in go testing that should be used more.
Lastly, the code
// lisper.go
// Package lisper turns a Go struct into a lisp style string, this is an alternative to the default
// stringer interface and works on types without them having to hand roll an output. For some use cases
// this makes for a more useful output.
package lisper
import (
"fmt"
"reflect"
"regexp"
"sort"
"strings"
)
// NodeList a list of nodes convert to string
type NodeList []string
func (nl NodeList) String() string {
return strings.Join(nl, " ")
}
// Lisper when implemented returns a slice of nodes and overrides the default conversion to a lisp node
// this is akin to the behaviour of the stringer interface
type Lisper interface {
LispString() []string
}
// Run convert a value to a lisp style string
func Run(v interface{}) NodeList {
return run(v)
}
// compare two arrays of strings
func compare(l []string, r []string) bool {
for i, j := range l {
if i > len(r) {
return true
}
if j > r[i] {
return false
}
}
return true
}
var quote = regexp.MustCompile("[,\"]")
// quoted replaces all quotes with escaped quotes and wraps the string in quotes
func quoted(n string, always bool) string {
if quote.Match([]byte(n)) {
return "\"" + strings.ReplaceAll(n, "\"", "\\\"") + "\""
} else if always {
return "\"" + n + "\""
} else {
return n
}
}
// run is the workhorse function, it is called recursively and returns a stack of lisp nodes
func run(v interface{}) []string {
// Check to see if the Lisper interface is implemented. If so return the custom node stack
if l, ok := v.(Lisper); ok {
return l.LispString()
}
// Create a value reference that doesn't worry if the passed in value is a pointer or not
val := reflect.Indirect(reflect.ValueOf(v))
typ := val.Type()
switch typ.Kind() {
case reflect.Interface, reflect.Struct:
{
res := []string{"("}
res = append(res, typ.Name())
for i := 0; i < typ.NumField(); i++ {
if typ.Field(i).IsExported() {
fieldValue := val.Field(i)
res = append(res, run(fieldValue.Interface())...)
}
}
res = append(res, ")")
return res
}
case reflect.Slice:
{
res := []string{"["}
for i := 0; i < val.Len(); i++ {
elemValue := val.Index(i)
res = append(res, run(elemValue.Interface())...)
}
res = append(res, "]")
return res
}
case reflect.Map:
{
// Keys are in a random order from a map, we sort the output to ensure stability
type kv struct {
key []string
value []string
}
kvs := make([]kv, val.Len())
for i, key := range val.MapKeys() {
value := val.MapIndex(key)
kvs[i] = kv{key: run(key.Interface()), value: run(value.Interface())}
}
sort.Slice(kvs, func(i, j int) bool {
return compare(kvs[i].key, kvs[j].key)
})
res := []string{"{"}
for _, kv := range kvs {
res = append(res, "(")
res = append(res, kv.key...)
res = append(res, ":")
res = append(res, kv.value...)
res = append(res, ")")
}
res = append(res, "}")
return res
}
case reflect.String:
{
node := fmt.Sprint(v)
return []string{quoted(node, true)}
}
default:
{
// Use the default string methods
node := fmt.Sprint(v)
return []string{quoted(node, false)}
}
}
}If you liked this walkthrough, please give it a thumbs up and subscribe if you want more of this sort of content.
