gno: gonative is going away

it's a system to connect gno and go code together, used for packages like fmt, but it has some problems and is not inspectable using static analysis

i often expressed in issue comments and conversations that i don't really like what i call "gonative". with that word, i generally refer to the entirety of the code in gnovm/pkg/gnolang/gonative.go, but also to the other large part of the codebase that handles the special cases of NativeType and NativeValue. there's been some efforts to improve it lately, and i've been reviewing and merging them provided that they 1. improve some existing flows 2. don't involve too much hassle in reviewing or further maintenance.

but, partly related to the native bindings effort, there's an issue to get these removed. to get some context, let's explore the two mechanisms whereby you can execute go code inside of gno:

• DefineNative. this is the underlying function used both for uverse functions, "package injections" and native bindings.
  • this is how uverse defines functions. uverse is the term we use for the "builtin" functions of gno, ie. append, len, cap, and so on. it directly calls DefineNative.
• native bindings essentially generates the native functions passed to DefineNative, which are generated by connecting a gno declaration with a go definition. here's an example of the generated code. this is how most native functions work now, like sha256.Sum256, or std.AssertOriginCaller().
  • note, this function uses Go2GnoValue, and Gno2GoValue. these are, in fact, from gonative; but they can eventually be removed with a better effort on code-generation, which would remove reflect from the equation here. (check this other blog post out for more code-generation in service of removing reflection.)
• NativeValue. this method allows to bind a go value directly to an equivalent gno symbol. these, at the time of writing, only exist when using the "testing context"; ie. gno run and gno test (but not on-chain). here's a list of current functions using it. the most useful function which uses NativeValue is fmt.Printf.

there are essentially two problems with NativeValue/NativeType:

static analysis

this is a problem that native bindings was attempting to solve as well: long ago, native functions were all defined in a big "native injector". the consequence was that none of the functions natively defined were "visible" from static analysis tools, like gno doc.

aside from gno doc, tools like gnopls cannot currently "see" these functions, without adding them as exceptions within the code itself. native bindings solved this by adopting the same approach used in go for functions that are defined using assembly: body-less declarations. if you inspect the sync/atomic package, as an example, you'll see that most functions are simply without a function body, because their source is in assembly.

similarly, in gno, native functions are now declared like this; to allow anyone who inspects the source to easily see the matching native go code.

go reflect limitations and risks

• while you can create new types using reflect, you cannot create new named types
  • as a consequence, no type you create with reflect can have methods.
  • as a consequence, no type you create with reflect can satisfy an interface. #3204
  • as a consequence, you cannot create "recursive types" (ie. which refer to themselves, imagine a linked list). #32205
• you can have a struct which has unexported fields - but you cannot actually set those unexported fields, due to reflect's limitations. #1155

this last one is particularly annoying, and is a known "gotcha" when testing; as it's common to be using unexported fields, but when you throw them into a fmt.Printf, the gnovm just panics. the "simple" solution is to use println instead, which, as a native machine function, has no problem with unexported fields. but it's still annoying.

the removal

in the short term, we're shrinking the amount of gonative code and removing its usage from the gnovm. by the mainnet launch, which is on the horizon, we should have all of its related code removed.

• for the os package, i expect there to be a testing-only shim that provides an interface to stdin / stdout.
• for the fmt and encoding/json package, i expect us to develop an mvp formatter and encoder which can be adequately be used in testing, likely in a restricted subset than their go counterparts, but which work directly on the internal TypedValue rather than trying to convert them to reflect values first.
• internal/os_test may simply be moved to be another variable which can be manipulated by the testing.Context struct, as part of the std re-organization
• math/big can probably be removed for the time being, awaiting a full implementation in gno.

there is nothing more satisfying than deleting code :)