CRUD API

Challenges

CRUD means short connecting to a database and Create, Read, Update and Delete records. This should not be hard, but as somebody pointed out there is no consensus about the "right way" to do this using Golang. In order to figure out how to do this, I have found some challenges and made some (good or bad?) decisions. Hence I have created this site in order to try to get some feedback.

Challenge #1: 1000+ endpoints / queries

Almost all videos and descriptions are at the "Hello World" level. Max 2 x 4 endpoints. Today I have about 100 SQL tables and x CRUD it means before starting the actual developing I have about 400 endpoints (100 tables * CRUD). This will be a real challenge.

Challenge #2: Maintain and manage for several years

One way to make the code maintainable is to reduce the lines of code. This means mostly not repeating almost the same code over and over. A rough calculation is that every endpoint consists of at least 10 rows of code. Meaning at least 10K of rows. Imagine that all code looks about the same, I find it a challenge to maintain. So I am searching for ways to make it easier to read. One way is to make the code as dynamic as possible. Not so popular with the Gophers.

Challenge #3: Increase speed

I am really tired of all the spinning wheels and "Loading..." messages. Using an admin software is boring enough. It should be superfast. So I am searching for the fastest possible tools and techniques as possible.

Challenge #4: Reduce flickering

When rendering a page there is always some flickering. So I am searching for techniques that reduce annoying flickering and delays in drawing.

Decisions

To achieve my goals I have to make many decisions. Good or bad. There are thousands of options to choose between. And within every option, there can be several ways to do the same thing.

Decision #1: Using Go to increase the speed

By using Go I have found a common ground of speed, simplicity and expandability. For now this is the main foundation for the rest of my decisions.

Decision #2: Using SSG to reduce flickering

Both SSR (Server Side Rendering) and CSR (Client Side Rendering) have their benefits. But sometimes SSG (Static Site Generation) may be more beneficial. Which means that Go renders the entire HTML at build time from templates in memory and are added to the page using innerHTML by Javascript.

Decision #3: NOT using ORM for maintainability reasons

JQuery is supposed to simplify Javascript. Instead it only adds another dependency layer and actually another language upon Javascript. This is about the same with ORM. Should simplify, but by creating another language upon SQL it makes it harder when it comes to advanced SQL queries. For example, I have found no ORM that supports the WITH statement.

Decision #4: Using sqlx in order to reduce maybe 1000 lines of Go code

The SQL lookup database holds all the information regarding a query. Structs repeating the column structure may be better in some cases, but creates tons of code lines with a potential risk of misspelling and errors. If there is a mismatch between the Struct and the table column struct I can imagine all hours that are spent to find all misspelling in the struct. Therefore I have decided to use sqlx, which gathers the struct information from the database query into a map[].

Decision #5: Store the SQL queries in a lookup database

If the endpoint (route) is the same as the query id, you can easily store 1000 queries and maintain them without restarting the Go executable. Using pgAdmin it is simple to add an endpoint and a SQL query. Besides this is way simpler to test the query in pgAdmin at the same time as you create the query. This reduces the code in Go, which is harder to maintain than pgAdmin.

Decision #6: Use dynamic endpoints for maintainability

Fixed endpoints must be a nightmare to maintain in a text based IDE. Either you have Loong code OR many separate packages. In my test with only 2 endpoints it has been hard to maintain. As the endpoint is the same as the stored SQL it is simple to find out if it is a legal endpoint or not.