PR #3847 implements the fmt package mostly in Gno, using a simplified reflection system; let's take a look at how it works
as part of my personal crusade to remove gonative, i recently made a pr that implements fmt using mostly gno, and a slight touch of reflection to get information about types and values dynamically.
these are a set of functions which are then implemented in go, using native bindings. in this article, we'll take a look at some of them to learn how they work, and how gno values can be created and handled under the hood!
at the core of the "micro-reflect" implementation, there is the valueOf function, returning a valueInfo - meant to be a substitute for reflect.ValueOf and reflect.Value. this is the gno code:
type valueInfo struct {
// Original value passed to valueOf.
Origin any
// Kind of the value.
Kind string
// if Origin is of a DeclaredType, the name of the type.
DeclaredName string
// Byte value for simple scalar types (integers, booleans).
Bytes uint64
// Base value stripped of the declared type.
Base any
// Length of the array, slice, string, struct, interface or map.
Len int
}
func valueOf(v any) valueInfo {
k, dn, bytes, base, xlen := valueOfInternal(v)
return valueInfo{v, k, dn, bytes, base, xlen}
}
this is implemented in go as follows:
func X_valueOfInternal(v gnolang.TypedValue) (
kind, declaredName string,
bytes uint64,
base gnolang.TypedValue,
xlen int,
) {
if v.IsUndefined() {
kind = "nil"
return
}
if dt, ok := v.T.(*gnolang.DeclaredType); ok {
declaredName = dt.String()
}
baseT := gnolang.BaseOf(v.T)
base = gnolang.TypedValue{
T: baseT,
V: v.V,
N: v.N,
}
switch baseT.Kind() {
case gnolang.BoolKind:
kind = "bool"
if v.GetBool() {
bytes = 1
}
case gnolang.StringKind:
kind, xlen = "string", v.GetLength()
case gnolang.IntKind:
kind, bytes = "int", uint64(v.GetInt())
case gnolang.Int8Kind:
kind, bytes = "int8", uint64(v.GetInt8())
case gnolang.Int16Kind:
// ...
case gnolang.ArrayKind:
kind, xlen = "array", v.GetLength()
case gnolang.SliceKind:
kind, xlen = "slice", v.GetLength()
case gnolang.PointerKind:
kind = "pointer"
case gnolang.StructKind:
kind, xlen = "struct", len(baseT.(*gnolang.StructType).Fields)
case gnolang.InterfaceKind:
kind, xlen = "interface", len(baseT.(*gnolang.InterfaceType).Methods)
case gnolang.FuncKind:
kind = "func"
case gnolang.MapKind:
kind, xlen = "map", v.GetLength()
default:
panic("unexpected gnolang.Kind")
}
return
}
this function returns base information about the value, which is passed in as a TypedValue. with these return values, we can do most of what we need in gno:
kind allows us to switch on the different, "base" kinds of the related values.base returns the value stripped of any declared type; this can allow us to do assertions on declared types which are actually strings, integers, or other kinds of values.bytes allows us to quickly gather a numeric value for all the numeric types, from boolean to int64 and float64.xlen (called as such to avoid shadowing) allows us to get the number of fields or elements in the slice, and through other functions then inspect the contents.as you can see, the code can be made relatively "unmagical" - the few things which may be less obvious are:
baseT := gnolang.BaseOf(v.T)
base = gnolang.TypedValue{
T: baseT,
V: v.V,
N: v.N,
}this is essentially a conversion from a declared type, if given, into its base value; for instance a time.Duration being converted into an int64.
the typed value itself is composed of three fields, as you may have seen. T and V are the type and value, while N is an 8-byte array used for all numeric and boolean values. this allows quick access to the underlying value when performing arithmetic - so under the hood, the calls to functions like GetInt8 are "cheap" - as they are simply literal type castings into the appropriate type of this field: *(*int8)(unsafe.Pointer(&tv.N)).
one important thing to note, is that in a TypedValue, the T can be nil (for the nil interface, which has neither a concrete type nor a nil value), as well as the value, for an unitialized value: a nil map, empty string, nil slice, nil function...; this is why we also need the guard at the beginning on IsUndefined, to ensure that a v.T exists and can be accessed.
in fmt, map keys are actually sorted using an internal package, internal/fmtsort. this allows us to compare multiple values, potentially also in a map where the key is an interface{}.
using fmtsort directly would mean implementing a lot of reflection to do so, so instead this is implemented directly by this internal function, which applies a similar algorithm directly on gno TypedValues.
then, we pass these values back into gno passing two []any, one for the keys and one for the values:
func X_mapKeyValues(v gnolang.TypedValue) (keys, values gnolang.TypedValue) {
if v.T.Kind() != gnolang.MapKind {
panic(fmt.Sprintf("invalid arg to mapKeyValues of kind: %s", v.T.Kind()))
}
keys.T = gSliceOfAny
values.T = gSliceOfAny
if v.V == nil {
return
}
mv := v.V.(*gnolang.MapValue)
ks, vs := make([]gnolang.TypedValue, 0, mv.GetLength()), make([]gnolang.TypedValue, 0, mv.GetLength())
for el := mv.List.Head; el != nil; el = el.Next {
ks = append(ks, el.Key)
vs = append(vs, el.Value)
}
// use stable to maintain the same order when we have weird map keys.
sort.Stable(mapKV{ks, vs})
keys.V = &gnolang.SliceValue{
Base: &gnolang.ArrayValue{
List: ks,
},
Length: len(ks),
Maxcap: len(ks),
}
values.V = &gnolang.SliceValue{
Base: &gnolang.ArrayValue{
List: vs,
},
Length: len(vs),
Maxcap: len(vs),
}
return
}i think there's two interesting things to consider here:
this function takes in a byte-ish array or slice and returns it as a byte slice. it is useful for cases like fmt.Sprintf("%x", sha256.Sum256("hey")) - because Sum256 returns an array, so we need something to convert it dynamically.
func X_asByteSlice(v gnolang.TypedValue) (gnolang.TypedValue, bool) {
switch {
case v.T.Kind() == gnolang.SliceKind && v.T.Elem().Kind() == gnolang.Uint8Kind:
return gnolang.TypedValue{
T: &gnolang.SliceType{
Elt: gnolang.Uint8Type,
},
V: v.V,
}, true
case v.T.Kind() == gnolang.ArrayKind && v.T.Elem().Kind() == gnolang.Uint8Kind:
arrt := v.T.(*gnolang.ArrayType)
return gnolang.TypedValue{
T: &gnolang.SliceType{
Elt: gnolang.Uint8Type,
},
V: &gnolang.SliceValue{
Base: v.V,
Offset: 0,
Length: arrt.Len,
Maxcap: arrt.Len,
},
}, true
default:
return gnolang.TypedValue{}, false
}
}perhaps this also helps to drive home how we can perform conversions effectively by changing the T in the TypedValue; while we always keep the same v.V, which effectively works directly on the raw values themselves.
this is how formatting works for some simple scalar values:
case "bool":
p.fmtBool(f.Bytes == 1, verb)
case "int", "int8", "int16", "int32", "int64":
p.fmtInteger(f.Bytes, signed, verb)
case "uint", "uint8", "uint16", "uint32", "uint64":
p.fmtInteger(f.Bytes, unsigned, verb)
case "float32":
p.fmtFloat(float64(math.Float32frombits(uint32(f.Bytes))), 32, verb)
case "float64":
p.fmtFloat(math.Float64frombits(uint64(f.Bytes)), 64, verb)
case "string":
p.fmtString(value.Base.(string), verb)as you can see, simply using valueOf allows us to already handle all of the scalar values easily.
here's how array and slice printing is implemented, instead:
case "array", "slice":
switch verb {
case 's', 'q', 'x', 'X':
if bs, ok := asByteSlice(f.Base); ok {
p.fmtBytes(bs, verb, typeString(f.Origin))
return
}
}
if p.fmt.sharpV {
p.buf.writeString(typeString(f.Origin))
if f.Kind == "slice" && getAddr(f.Base) == 0 {
p.buf.writeString(nilParenString)
return
}
p.buf.writeByte('{')
for i := 0; i < f.Len; i++ {
if i > 0 {
p.buf.writeString(commaSpaceString)
}
p.printValue(valueOf(arrayIndex(f.Base, i)), verb, depth+1)
}
p.buf.writeByte('}')
} else {
p.buf.writeByte('[')
for i := 0; i < f.Len; i++ {
if i > 0 {
p.buf.writeByte(' ')
}
p.printValue(valueOf(arrayIndex(f.Base, i)), verb, depth+1)
}
p.buf.writeByte(']')
}typeString is another native function that allows to get the string representation, using the same methods that are used in error messages and such.
i think ther's room for improvement, but working on this tiny reflection system gave me hope that a full "micro-reflect" package is something we can see sometime soon in gno :)
mainnet work is keeping me busy, so this blog hasn't seen much activity lately! but rest assured there's going to be more content coming. i also spent much of my writing efforts creating a strong piece of technical documentation of the gnovm, to be used as a readme.