Extensions

The collection/key/value architecture of YapDatabase is just the foundation for an advanced plugin system. These plugins are called "extensions". An extension provides additional functionality beyond that provided by the collection/key/value API.

YapDatabase ships with several useful extensions, and you can write your own as well. The following extensions are available:

• YapDatabaseAutoView
• YapDatabaseManualView
• YapDatabaseFilteredView

• YapDatabaseSecondaryIndex
• YapDatabaseFullTextSearch
• YapDatabaseSearchResultsView
• YapDatabaseRelationship
• YapDatabaseRTreeIndex
• YapDatabaseHooks
• YapDatabaseActionManager
• YapDatabaseCloudKit
• YapDatabaseCloudCore

 

Overview

Extensions work rather seamlessly as they were designed to match the existing architecture of YapDatabase.

The extensions architecture was designed around 3 core principles:

• Automatic
• Transactional
• Persistent

 

Automatic

One of the best features of extensions is that they automatically update themselves. Let's look at a concrete example. First, this is what your code looks like before you start using extensions:

databaseConnection.readWrite {(transaction) in
  // without extensions:
  transaction.setObject(myObject, forKey:"abc123", inCollection:"example")
}

And this is what your code looks like after you start using extensions:

databaseConnection.readWrite {(transaction) in
  // with extensions:
  transaction.setObject(myObject, forKey:"abc123", inCollection:"example")
  // nothing else to do...
  // the extensions are automatically notified of the change,
  // and automatically updated.
}

As you can see, there is no difference. So if you already have a bunch of code, you don't have to worry about making a ton of changes throughout your application.

Once you register an extension, it is automatically forwarded any changes you make to the database. Essentially, there are internal "hook" methods that extensions implement. These hook methods correspond to the existing read-write API. So, from the example above, a call to setObject(_:forKey:inCollection:) is automatically forwarded to all registered extensions. This allows extensions to automatically keep their data up-to-date.

 

Transactional

The transaction is a core component of YapDatabase. As such, extensions are fully integrated into the transactional architecture.

That is, if you start a read-only transaction, you know you have a "snapshot-in-time" of the database at that moment. That is, you're viewing the database at a particular commit number. The data will not change during your transaction, even if a background thread is making changes. This is the power of transactions. And extensions work the same way.

In fact, you access your extension(s) in the exact same way you're used to accessing the database: through a transaction

databaseConnection.read {(transaction) in

  if let viewTransaction = transaction.ext("SalesByCountry") as? YapDatabaseViewTransaction {
    topUsaSale = viewTransaction.object(atIndex:0, inGroup:"US")
  }
}

The same goes for read-write transactions. So if you make changes to the database, you know those changes, and any changes that may have automatically occurred to extensions, are all part of the atomic read-write transaction.

So extensions are transactional and atomic, just like the database is.

 

Persistent

Extensions have full access to the sqlite internals. So an extension may create it's own sqlite table(s). Or perhaps it uses an existing sqlite module (which may in turn create its own virtual table).

The point is this: The data that an extension writes to the database is persistent, just like the rest of the key/value API.

So an extension doesn't have to re-create its data on every app launch (assuming you don't change its configuration). It may simply read from its existing internal sqlite table.

Note: Its also legal to create extensions that reside fully in memory. It all depends on what the extension is trying to achieve.

 

---

Registration

Extensions are completely optional. You can use zero, or one, or multiple extensions. You can even use multiple instances of the same extension class. It's all up to you.

In order to use an extension, you must go through the registration process. Here's how it works.

1. Create an instance of your extension (configured however you like).

   let salesView = YapDatabaseView(/*...*/)

2. Register the extension instance with the database. When you do this, you choose a name for the extension instance. This name can be whatever you want. (Recall that it's possible to register multiple instances of the same extension class.)

   database.registerExtension(salesView, withName:"sales")

3. Once registered, the extension can be accessed from within any transaction by simply using the registered name.

   databaseConnection.read {(transaction) in
     
     if let viewTransaction = transaction.ext("sales") as? YapDatabaseViewTransaction {
       topUsaSale = viewTransaction.object(atIndex:0, inGroup:"US")
     }
   }

As you can imagine, each extension provides a number of new API's that allow you to interact with it. The YapDatabaseView extension (in the example above) provides a persistent "view" of the data. That is, imagine a SQL query which does something like:

SELECT ... FROM database
WHERE (filtering criteria...)
GROUP BY (grouping criteria...)
ORDER BY (sorting criteria...)

You provide the filtering, grouping and sorting criteria to the view (with Swift code!), and it handles all the rest. It automatically updates itself when you make changes to the database. And it tells you exactly what changes occurred within the view (such as index 5 moved to index 3).

 

Re-Registration

Registering your extension(s) is something you do everytime you initialize the database. If you haven't made any changes to the configuration of the extension since last app run, then the extension will register very quickly.

If you have made changes to the configuration of the extension, then it may need to update itself. For example, say you have a view, and you've decided to change the sorting criteria. So you make the appropriate changes to the view's sortingBlock, increment the view's version number (to signal a configuration change), and relaunch the app.

(More details on configuring and updating views are available on the Views page.)

The view will see that its configuration has been changed, and it will go about updating itself. That is, it will dump its internal tables, and repopulate itself.

As such, it's recommended that you always use async registration (so the disk IO doesn't slow down your app launch):

database.asyncRegisterExtension(salesView, withName:"sales"){(ready) in

  print("The 'sales' view is updated, and ready for use, using the updated sorting criteria.")
}

And you don't have to tiptoe around the database while extensions are registering and updating themselves. The database can still be read from (but obviously your extension won't be accessible until its completed the registration process). You can think of the registerExtension method as a readWrite transaction, and asyncRegisterExtension as an asyncReadWrite transaction.

 

On-The-Fly Registration

You can register an extension at any point while using the database. For example, you may opt to create a view only for a particular viewController.

As part of the registration process, the view will enumerate over the existing rows in the database, and automatically populate itself according to its configuration.

The database keeps track of which extensions are registered, under which names, what classes they're using, and whether or not they're up-to-date. Long story short, the database knows if you're registering an extension for the first time, or simply re-registering it from a previous app launch. The extensions use this information, which allows them to re-register quickly, or automatically populate themselves when needed.

 

On-The-Fly Unregistration

Just as you can bring up extensions on-the-fly, you can take them down on-the-fly.

Unregistering an extension removes it from the database, and results in the associated underlying sqlite table(s) for the extension being dropped.

database.asyncUnregisterExtension("sales")

 

Automatic Unregistration

What if I no longer need an extension? Like, I used it in version 1 of my app. But I'm doing something different in version 2, and I don't need it anymore. I know I can simply not register the extension anymore, but I want the database to delete the underlying table(s) the extension was using. Do I have to create the instance, and register it, just so I can unregister it?

Actually, you don't have to do a thing. Just don't register the extension anymore and the tables will be automatically deleted.

Here's how it works:

When you register an extension, the database records the name you used to register the extension, along with the class name of the extension.

When you unregister an extension, you simply pass the registered name. At this point, YapDatabase will invoke a class method on the associated class to delete the associated tables. So as long as YapDatabase can still find the class associated with the extension, it can delete any associated underlying sqlite tables.

If you want to be explicit about it, you can manually invoke the unregisterExtension: method. All you'd have to do is pass it the name you were using back in version 1 of your app, and it will then use the corresponding extension class to execute the code to delete the tables ("DROP TABLE IF EXISTS ...").

However, all this will happen automatically for you. YapDatabase knows what extensions you had previously registered. So if you don't re-register them on a subsequent app launch, and then proceed to make modifications to the database, YapDatabase will recognize that the extension(s) are now out-of-sync with the database, and will proceed to delete them.

YapDatabase will automatically drop the sqlite table(s) associated with orphaned extensions within the first read-write transaction that makes changes to the database.

Long story short: just stop registering the extension, and YapDatabase will automatically do the right thing.