ESC
Type to search...
S
Soli Docs

Database Migrations

Evolve your database schema with versioned migration files. Create collections, indexes, and manage schema changes safely.

Overview

Migrations are stored in db/migrations/ with timestamped filenames. Each migration contains up() and down() functions for applying and rolling back changes.

Note: Collections are now automatically created when you first use a Model. You can start using your models immediately without creating migrations. However, for production applications, we recommend using migrations to define indexes for better query performance, set collection options, and document your schema.

def up(db: Any)
  db.create_collection("users")
  db.create_index("users", "idx_email", ["email"], { "unique": true })
end

def down(db: Any)
  db.drop_index("users", "idx_email")
  db.drop_collection("users")
end

Which backends? Everything on this page is written for SoliDB, the default. Migrations also run on the SQL adapters (postgres, mysql, sqlite), where they build document tables and tables with real columns — see On the SQL adapters.

Auto-loaded models

Migrations run with your app/models (and app/services) auto-loaded — the same recursive walk as soli serve and db:seed. Model classes are available by name without an import, so data migrations can use the Model API:

def up(db)
  for user in User.all()
    next unless user.slug.blank?
    user.slug = user.name.downcase().gsub(" ", "-")
    user.save()
  end
end

def down(db)
  # irreversible backfill
end

This works on every backend — SoliDB and the SQL adapters alike. Schema work still uses the db handle (create_collection / create_table, create_index, query, …). Prefer db.* for pure schema changes and the Model API when you need validations, callbacks, or associations during a data migration.

CLI Commands

generate Create a Migration

soli db:migrate generate create_users_table

Creates: db/migrations/20260122143052_create_users_table.sl

up Run Migrations

# Apply all pending migrations
soli db:migrate

# Or explicitly
soli db:migrate up

# On a SQL adapter, create the database first — SoliDB
# makes its own on first use, a SQL server does not:
soli db:create
soli db:schema:dump   # write db/schema.sql + applied versions
soli db:schema:load   # recreate a fresh database from that dump
soli db:drop     # removes it (WAL sidecars too, on SQLite)

All four refuse an unknown or value-less flag with exit 64 rather than ignoring it. That matters most for db:drop: soli db:drop --connection with the value forgotten used to fall through with no connection and drop the default database without a word. An in-file connection "name" is likewise resolved against config/database.toml at load time — a typo used to print Applied while migrating the default SoliDB instead.

down Rollback

# Rollback the last migration
soli db:migrate down

status Check Status

soli db:migrate status
  Database Migrations

  Version         Name                            Status
  --------------  ------------------------------  ----------
  20260122143052  create_users_table                 up
  20260122145201  add_posts_table                    up
  20260122151033  add_user_indexes                  down

  2 applied, 1 pending

Collection Helpers

db.create_collection(name, type?)

Create a collection. type is optional — one of "blob", "edge", "timeseries", … (forwarded verbatim to SolidB). Default is a document collection.

  • blob — binary attachments; required for solidb_store_blob and the uploader DSL.
  • edge — graph edges (_from/_to); backs the edge model DSL and traversals.
  • timeseries — append-only time-indexed events (metrics, logs, telemetry); backs the timeseries model DSL.
  • "columnar" now raises — columnar stores are not document collections (it used to silently create a mislabeled document collection). Use db.create_columnar instead.
db.create_columnar(name, columns, options?)

Create a columnar store. columns is an array of { "name": ..., "type": ..., "nullable"?: bool, "indexed"?: bool } hashes; options accepts { "compression": "lz4" | "none" } (default lz4).

db.drop_columnar(name)

Remove a columnar store and all its data.

db.prune_collection(name, cutoff)

Delete documents older than an RFC3339 cutoff from a timeseries collection — the migration-side counterpart of Model.prune.

db.drop_collection(name)

Remove a collection and all its data.

db.list_collections()

List all collections in the database.

db.collection_stats(name)

Get statistics for a collection.

def up(db: Any)
  db.create_collection("users")                       # document
  db.create_collection("posts")
  db.create_collection("contact_documents", "blob")   # blob
  db.create_collection("follows", "edge")             # edge (graph)
  db.create_collection("metrics", "timeseries")       # timeseries

  # Columnar stores use the dedicated helper (create_collection with
  # "columnar" raises — columnar stores are not document collections)
  db.create_columnar("page_views", [
    { "name": "url", "type": "string" },
    { "name": "country", "type": "string", "indexed": true }
  ])

  # Edge collections: hash-index the endpoints so traversals stay fast
  # (dev auto-create does both steps for you)
  db.create_index("follows", "idx_follows_from", ["_from"], {})
  db.create_index("follows", "idx_follows_to", ["_to"], {})

  # Timeseries collections: one-off cleanup of old rows
  db.prune_collection("metrics", "2026-01-01T00:00:00Z")
end

def down(db: Any)
  db.drop_collection("comments")
  db.drop_collection("posts")
  db.drop_collection("users")
end

Index Helpers

db.create_index(collection, name, fields, options)

Create an index on a collection.

  • fields - Array of field names: ["email"] or ["first_name", "last_name"]
  • options - Hash with unique: true and/or type: — "hash" (default), "persistent", "fulltext", "bloom", or "cuckoo" ("skiplist"/"btree" are aliases for "persistent")
  • The old sparse option was dropped — the server never read it, so it changed nothing.
db.create_vector_index(collection, name, field, dimension, options?)

Create an HNSW vector index for ANN search. options is a metric string ("cosine", default) or a hash with metric and quantization. Drop with db.drop_vector_index(collection, name).

db.drop_index(collection, name)

Remove an index from a collection.

db.list_indexes(collection)

List all indexes for a collection.

Models can also declare indexes in the class body (index, vector_index, fulltext_index, geo_index — see Search). Declarations are metadata-only: dev creates them at server boot; in production mirror them in migrations (the recommended DDL path) or run the reconciler soli db:indexes [folder]. Geo indexes currently have no migration helper — soli db:indexes (or dev boot) creates them.

def up(db: Any)    # Simple index
  db.create_index("users", "idx_email", ["email"], {})
  # Unique index
  db.create_index("users", "idx_username", ["username"], { "unique": true })
  # Typed index — "hash" is the default; "persistent" for sorted/range lookups
  db.create_index("users", "idx_age", ["age"], { "type": "persistent" })
  # Fulltext index
  db.create_index("articles", "idx_articles_ft", ["title", "body"], { "type": "fulltext" })
  # Compound index on multiple fields
  db.create_index("users", "idx_name", ["first_name", "last_name"], {})
  # Unique compound index
  db.create_index("posts", "idx_user_slug", ["user_id", "slug"], { "unique": true })
end

def down(db: Any)
  db.drop_index("posts", "idx_user_slug")
  db.drop_index("users", "idx_name")
  db.drop_index("articles", "idx_articles_ft")
  db.drop_index("users", "idx_age")
  db.drop_index("users", "idx_username")
  db.drop_index("users", "idx_email")
end

Raw Queries

For operations not covered by helpers, use raw SDBQL queries:

def up(db: Any)    # Insert seed data
  db.query("INSERT { name: 'Admin', role: 'admin' } INTO users")
  # Update existing data
  db.query("FOR u IN users FILTER u.role == 'guest' UPDATE u WITH { role: 'user' } IN users")

  # Bind variables (preferred for user data — avoids escaping issues)
  digest = bcrypt_hash("changeme")
  db.query(
    "INSERT { email: @e, name: @n, role: @r, password_digest: @d } INTO users",
    { "e": "admin@example.com", "n": "Admin", "r": "admin", "d": digest }
  )

  # Bind variables work in FILTER / RETURN too
  db.query(
    "FOR doc IN users FILTER doc.status == @status RETURN doc",
    { "status": "active" }
  )
end

Raw queries are also the escape hatch for SolidB features the helpers don't wrap — e.g. continuous aggregates over a timeseries collection via db.query("CREATE STREAM ...").

Complete Example

A migration for a blog application:

def up(db: Any)    # Create collections
  db.create_collection("users")
  db.create_collection("posts")
  db.create_collection("comments")
  db.create_collection("tags")
  # User indexes
  db.create_index("users", "idx_users_email", ["email"], { "unique": true })
  db.create_index("users", "idx_users_username", ["username"], { "unique": true })
  # Post indexes
  db.create_index("posts", "idx_posts_author", ["author_id"], {})
  db.create_index("posts", "idx_posts_slug", ["slug"], { "unique": true })
  db.create_index("posts", "idx_posts_published", ["published_at"], { "sparse": true })
  # Comment indexes
  db.create_index("comments", "idx_comments_post", ["post_id"], {})
  db.create_index("comments", "idx_comments_author", ["author_id"], {})
  # Tag indexes
  db.create_index("tags", "idx_tags_name", ["name"], { "unique": true })
end

def down(db: Any)    # Drop indexes first
  db.drop_index("tags", "idx_tags_name")
  db.drop_index("comments", "idx_comments_author")
  db.drop_index("comments", "idx_comments_post")
  db.drop_index("posts", "idx_posts_published")
  db.drop_index("posts", "idx_posts_slug")
  db.drop_index("posts", "idx_posts_author")
  db.drop_index("users", "idx_users_username")
  db.drop_index("users", "idx_users_email")
  # Drop collections
  db.drop_collection("tags")
  db.drop_collection("comments")
  db.drop_collection("posts")
  db.drop_collection("users")
end

Seeding the Database

Migrations build the schema; seeds populate it with data — demo accounts, lookup tables, an initial admin user. Run them with soli db:seed.

Unlike migrations, seeds are not tracked — every file runs on every invocation. Make your seeds idempotent (guard with first_by / find_by) so re-running them doesn't create duplicates.

Seeds live in two places, run in this order:

db/seeds.sl          # runs first
db/seeds/*.sl        # then every file here, sorted by name

generate Create a Seed File

soli db:seed generate demo_users

Creates: db/seeds/20260623161240_demo_users.sl

seed Run Seeds

# Run db/seeds.sl, then db/seeds/*.sl
soli db:seed

# Run a single seed file (path relative to the project folder)
soli db:seed db/seeds/20260623161240_demo_users.sl

# Point at a different project folder
soli db:seed ./myapp

Seeds run with your app/models (and app/services) auto-loaded, so they can use the Model API directly — no imports needed:

3.times do |i|
  let email = "user#{i}@example.com"
  User.create({ "name": "User #{i}", "email": email }) if User.first_by("email", email).nil?
end

print("Seeded users")

Targeting a connection

A migration can name the database it belongs to, so soli db:migrate up places it correctly with no CLI flag.

connection "analytics"

def up(db)
  db.create_table("events", { "id": "pk", "name": "string" })
end

def down(db)
  db.drop_table("events")
end
  • The declaration must be the first non-comment statement (blank lines and # / // comments may precede it). A connection "…" line inside a string, or after def up, is ignored. A second declaration is an error, not a silent first-wins. The runner reads it before running anything, and it never executes as a statement.
  • Each connection tracks its own versions — a migration applied to analytics is not marked applied on the default connection.
  • Without a declaration, a migration runs on --connection if given, else the default connection.
  • --connection NAME is a filter: a migration declaring another database is held back (and reported as skipped) rather than applied to the wrong schema.
  • soli db:migrate status grows a Connection column as soon as one migration declares one.

On the SQL adapters

On a postgres, mysql, or sqlite connection a migration can build either kind of table: a document table (_key + doc) from create_table("posts") with no column hash, or a column table from create_table("orders", { … }) — the tables column-aware models map onto. Anything SoliDB-specific raises rather than being silently skipped. Per-adapter notes: PostgreSQL, MySQL, SQLite.

Column tables

def up(db)
  db.create_table("orders", {
    "id":         "pk",
    "code":       { "type": "string", "limit": 32, "null": false },
    "amount":     "decimal(10,2)",
    "qty":        "integer",
    "paid":       { "type": "boolean", "default": false },
    "meta":       "json",
    "user_id":    { "type": "bigint", "references": "users" },
    "timestamps": true
  })
  db.add_index("orders", ["code"], { "unique": true })
end

def down(db)
  db.drop_table("orders")
end

One migration, three backends: the types are Soli's and each adapter renders its own SQL. The rendered names are chosen so introspection reads the table back as the same Soli types — a table created this way is always one a column-aware model can map.

Soli type Postgres MySQL SQLite
pkBIGSERIAL PRIMARY KEYBIGINT AUTO_INCREMENT PRIMARY KEYINTEGER PRIMARY KEY AUTOINCREMENT
uuid_pkUUID PRIMARY KEYCHAR(36) PRIMARY KEYUUID PRIMARY KEY
string / string(n)VARCHAR(255) / VARCHAR(n)samesame
textTEXTTEXTTEXT
integer / bigintINTEGER / BIGINTINT / BIGINTINTEGER / BIGINT
floatDOUBLE PRECISIONDOUBLEREAL
decimal(p,s)NUMERIC(p,s)DECIMAL(p,s)DECIMAL(p,s)
booleanBOOLEANTINYINT(1)BOOLEAN
date / datetimeDATE / TIMESTAMPTZDATE / DATETIMEDATE / DATETIME
jsonJSONBJSONJSON
uuid / binaryUUID / BYTEACHAR(36) / BLOBUUID / BLOB
  • Column options: type, limit (string only), null, unique, primary_key, default, references.
  • "timestamps": true adds created_at / updated_at as NOT NULL DEFAULT CURRENT_TIMESTAMP.
  • "references": "users" points at that table's id; write "users(uuid)" to name another column. On MySQL the constraint is emitted at table level, because MySQL parses an inline REFERENCES and then ignores it.

Schema helpers

Helper Notes
db.create_table(name)Document table (_key + doc)
db.create_table(name, columns)Column table, as above
db.drop_table(name)Either kind
db.add_column(table, name, type)Type string or options hash
db.drop_column(table, name)
db.rename_column(table, old, new)MySQL 8+
db.rename_table(old, new)
db.add_index(table, columns, options?){ "unique": true, "name": "…" }; name defaults to idx_<table>_<columns>
db.add_index(table, ["doc.status"])A doc. prefix indexes a JSON field of a document table (expression index; a generated column on MySQL)
db.drop_index(table, name)
db.create_index(table, name, fields, options?)The SoliDB-shaped call, so a shared migration keeps working
db.execute(sql)Escape hatch — engine-specific by definition. Migration-only (not callable from controllers, jobs, or templates). Runs on a dedicated connection so SET / ATTACH / PRAGMA cannot leak into the request pool.
db.create_collection / db.drop_collectionAliases for document tables
db.create_collection(name, "edge"|…)✗ raises — typed collections are SoliDB-only
db.query(sdbql)✗ raises — SDBQL has no meaning on SQL
db.create_columnar, db.create_vector_index✗ raise — SoliDB-only

Two limits worth knowing before you hit them. SQLite's ALTER TABLE is narrow — it cannot add a UNIQUE or primary-key column to an existing table (add the column, then add_index), and it cannot add a NOT NULL column without a default; both are reported as errors naming the way around them. And changing a column's type is not portable, so there is no change_column: use db.execute, or add-copy-drop. _migrations, _jobs, and _cron_jobs are reserved — a migration that creates, drops, or renames them is refused.

Models never issue DDL. A column-aware model maps to a table it does not own: no auto-create, no index sync, no implicit ALTER. Migrations are the one place Soli changes a column table, and only where you wrote it.

Environment Configuration

Configure database connection via .env file:

SOLIDB_HOST=http://localhost:6745
SOLIDB_DATABASE=myapp_development
SOLIDB_USERNAME=root
SOLIDB_PASSWORD=secret

Best Practices

  • Keep migrations small - One logical change per migration
  • Always write down() - Enable clean rollbacks
  • Test rollbacks - Run down then up to verify
  • Order matters in down() - Drop indexes before collections
  • Don't modify old migrations - Create new ones for changes

Helpers Reference

Method Description
db.create_collection(name, type?) Create a collection. type is optional — "blob", "edge", "timeseries", … default is document. "columnar" raises — use create_columnar.
db.create_columnar(name, columns, options?) Create a columnar store. columns: array of {name, type, nullable?, indexed?} hashes; options: {"compression": "lz4"|"none"}
db.drop_columnar(name) Drop a columnar store
db.prune_collection(name, cutoff) Delete documents older than an RFC3339 cutoff from a timeseries collection
db.drop_collection(name) Drop a collection
db.list_collections() List all collections
db.collection_stats(name) Get collection statistics
db.create_index(collection, name, fields, options) Create an index. options: unique: and type: ("hash" default, "persistent", "fulltext", "bloom", "cuckoo")
db.create_vector_index(collection, name, field, dimension, options?) Create an HNSW vector index. options: metric string or {metric, quantization} hash
db.drop_vector_index(collection, name) Drop a vector index
db.drop_index(collection, name) Drop an index
db.list_indexes(collection) List indexes for a collection
db.query(sdbql, bind_vars?, options?) Execute a raw SDBQL query. options accepts { "timeout": secs }; db.timeout(secs) sets it on the client

Next Steps