extending search on a model

[ivang76]

Hi there, I'm trying to extend the internal search to work also on a model. I've followed the instructions found here:
https://statamic.dev/extending/search
but it doesn't work, I have no results... someone have any ideas about what could be the cause?

My model implements the searchable:

<?php

namespace App\Models;

use Illuminate\Database\Eloquent\Factories\HasFactory;
use Illuminate\Database\Eloquent\Model;
use DoubleThreeDigital\Runway\Traits\HasRunwayResource;

use Statamic\Contracts\Search\Result as ResultContract;
use Statamic\Contracts\Search\Searchable as SearchableContract;
use Statamic\Search\Searchable;
use Statamic\Search\Result;

class Feed extends Model implements SearchableContract
{
    use HasFactory;
    use HasRunwayResource; 
    use Searchable;


    /**
    * The identifier that will be used in search indexes.
    */
   public function getSearchReference(): string
   {
       return 'feed::'.$this->id;
   }

   /**
    * The indexed value for a given field.
    */
   public function getSearchValue(string $field)
   {
       return $this->$field;
   }

   /**
    * Convert to a search result class.
    * You can use the Result class, or feel free to create your own.
    */
   public function toSearchResult(): ResultContract
   {
       return new Result($this, 'feed');
   }


}

I have a class that extends a provider:

<?php
namespace App\Search\Searchables;

use Statamic\Search\Searchables\Provider;
use App\Models\Feed;
use Illuminate\Support\Collection;


class FeedProvider extends Provider
{
    /**
     * The handle used within search configs.
     *
     * e.g. For 'searchables' => ['collection:blog', 'products:hats', 'users']
     *      they'd be 'collection', 'products', and 'users'.
     */
    protected static $handle = 'feeds';
 
    /**
     * The prefix used in each Searchable's reference.
     *
     * e.g. For 'entry::123', it would be 'entry'.
     */
    protected static $referencePrefix = 'feed';
 
    /**
     * Convert an array of keys to the actual objects.
     * The keys will be searchable references with the prefix removed.
     */
    public function find(array $keys):Collection
    {
        info('Find method called with keys:', $keys);
        return Feed::find($keys);
    }
 
    /**
     * Get a collection of all searchables.
     */
    public function provide(): Collection
    {
        info('Provide method called');
        return Feed::all();
 
        // If you wanted to allow subsets of products, you could specify them in your
        // config then retrieve them appropriately here using keys.
        // e.g. 'searchables' => ['products:hats', 'products:shoes'],
        // $this->keys would be ['keys', 'hats'].
        // return Feed::whereIn('type', $this->keys)->get();
    }
 
    /**
     * Check if a given object belongs to this provider.
     */
    public function contains($searchable): bool
    {
        info('contains method called');
        return $searchable instanceof Feed;
    }


}

The provider is registered

public function boot()
{
    FeedProvider::register();
}

and the search config is updated:

    'indexes' => [

        'default' => [
            'driver' => 'local',
            'searchables' => 'all',
            'fields' => ['title'],
        ],

        'feeds' => [
            'driver' => 'local',
            'searchables' => [
                'feeds'
            ],
            'fields' => ['body_it','body_en']
        ]
    ],

I debugged and placed some log traces and I notice that the FeedProvider methods are not executed during a search even if I'm sure that the FeedProvider::register() is called

[duncanmcclean]

I see you're using Runway - is there any reason you're not using Runway's built-in search functionality?

[ivang76]

Not really. I just tested Runway to manage the model in the CP and didn't know that I can also use it for the search. Thank you; I'll definitely try it. However, I would also like to understand how to extend the search functions using only Statamic.

[ivang76]

I quickly tested also with runway but I still get no results

        'feeds_runway' => [
            'driver' => 'local',
            'searchables' => ['runway:*'],
            'fields' => ['title_en']
        ]
        
        
    'resources' => [
        \App\Models\Feed::class => [
            'name' => 'Feeds',
            'blueprint' => 'feeds',
            'title_field' => 'title_en',
        ],          

[duncanmcclean]

Hmm, this seems to be working for me too... here's my code for clarity.

I'm guessing you've run php please search:update and the data is being populated in storage/statamic/search? How are you attempting to search?

[duncanmcclean]

I've just ran through the custom searchables documentation quickly and was able to get a custom searchable working. For clarity, here's all my search-related code.

I think the only difference is that I had to add methods to handle augmentation, whereas your model doesn't have any of those models (Runway handles augmentation for you via it's HasRunwayResource trait).

A couple of questions for you:

• After configuring all the search stuff, I'm guessing you ran php please search:update? (sounds like a silly question but want to double check)
• When using the local driver, Statamic will store the indexed data in JSON files in the storage/statamic/search directory. Can you open up the relevant JSON file for your feeds index and see if it's being populated with the relevant fields?
• How are you attempting to search the models? Using the {{ search:results }} tag or in the Control Panel? If you're using the tag, can you share your template?

[ivang76]

First of all thanks for your time and effort...
yes I run the php please search:update and I see the json files under the storage/statamic/search.

I'm using a simple antlers view to show the result for now (later I would like to use a typical controller + blade flow):

{{ search:results index="default" }}
    {{ if no_results }}
        <h2>No results found for {{ get:q }}.</h2>
    {{ else }}
        <a href="{{ url }}">
            <h2>{{ title }}</h2>
        </a>
    {{ /if }}
{{ /search:results }}

{{ search:results index="feeds" }}
    {{ if no_results }}
        <h2>No results found for {{ get:q }}.</h2>
    {{ else }}
        <a href="{{ url }}">
            <h2>{{ title_en }}</h2>
        </a>
    {{ /if }}
{{ /search:results }}

{{ search:results index="feeds_runway" }}
    {{ if no_results }}
        <h2>No results found for {{ get:q }}.</h2>
    {{ else }}
        <a href="{{ url }}">
            <h2>{{ title_en }}</h2>
        </a>
    {{ /if }}
{{ /search:results }}

No results at all... nothing is found on the Feed model with both the ways

[ivang76]

Hi again. I attempted to import and copy your 'post' example, but I encountered an error when running a search:

Searchable App\Models\Post must implement Statamic\Contracts\Query\ContainsQueryableValues

it is strange that you didn't get the same error, anyway I updated your model to match mine related to 'feeds,' adding what is required (QueryableValues). Here's the new version:

<?php

namespace App\Models;

use Illuminate\Database\Eloquent\Factories\HasFactory;
use Illuminate\Database\Eloquent\Model;
use Statamic\Contracts\Data\Augmentable;
use Statamic\Contracts\Data\Augmented;
use Statamic\Contracts\Search\Result as ResultContract;
use Statamic\Contracts\Search\Searchable as SearchableContract;
use Statamic\Data\AugmentedData;
use Statamic\Search\Result;
use Statamic\Search\Searchable;
use Statamic\Contracts\Query\ContainsQueryableValues;

class Post extends Model implements Augmentable, SearchableContract, ContainsQueryableValues
{
    use HasFactory, Searchable;

    protected $fillable = [
        'name', 'slug',
    ];

    /**
     * The identifier that will be used in search indexes.
     */
    public function getSearchReference(): string
    {
        return 'post::'.$this->id;
    }

    /**
     * The indexed value for a given field.
     */
    public function getSearchValue(string $field)
    {
        return $this->$field;
    }

    /**
     * Convert to a search result class.
     * You can use the Result class, or feel free to create your own.
     */
    public function toSearchResult(): ResultContract
    {
        return new Result($this, 'post');
    }

    // augmentation
    public function augmentedValue($key)
    {
        return $this->augmented()->get($key);
    }

    public function toAugmentedCollection($keys = null)
    {
        return $this->augmented()
            ->withRelations($this->defaultAugmentedRelations())
            ->select($keys ?? $this->defaultAugmentedArrayKeys());
    }

    public function toAugmentedArray($keys = null)
    {
        return $this->toAugmentedCollection($keys)->all();
    }

    public function toShallowAugmentedCollection()
    {
        return $this->augmented()->select($this->shallowAugmentedArrayKeys())->withShallowNesting();
    }

    public function toShallowAugmentedArray()
    {
        return $this->toShallowAugmentedCollection()->all();
    }

    public function augmented()
    {
        return $this->newAugmentedInstance();
    }

    protected function defaultAugmentedArrayKeys()
    {
        return null;
    }

    public function shallowAugmentedArrayKeys()
    {
        return ['id', 'title', 'api_url'];
    }

    protected function defaultAugmentedRelations()
    {
        return [];
    }

    public function toEvaluatedAugmentedArray($keys = null)
    {
        $collection = $this->toAugmentedCollection($keys);

        // Can't just chain ->except() because it would return a new
        // collection and the existing 'withRelations' would be lost.
        if ($exceptions = $this->excludedEvaluatedAugmentedArrayKeys()) {
            $collection = $collection
                ->except($exceptions)
                ->withRelations($collection->getRelations());
        }

        return $collection->withEvaluation()->toArray();
    }

    protected function excludedEvaluatedAugmentedArrayKeys()
    {
        return null;
    }

    public function newAugmentedInstance(): Augmented
    {
        return new AugmentedData($this, $this->augmentedArrayData());
    }

    public function augmentedArrayData()
    {
        return method_exists($this, 'values') ? $this->values() : $this->toArray();
    }

    public function queryableValues()
    {
        // Define the queryable values here.
        return [
            'name' => $this->name,
        ];
    }

    public function getQueryableValue(string $key)
    {
        return $this->$key;
    }


}

ok, now no more error, but still no results found... any ideas? thanks!

[duncanmcclean]

Hmm, I'm not quite sure why it's not working.... 🤔

Maybe you could try changing any references to title from my code to the title_en column you're using instead?

If you're able to, are you able to replicate the same issues on a fresh Statamic site?

[ivang76]

I have around another installation with this example:
https://statamic.com/starter-kits/statamic/multisimplicity

so I linked the same db and tried to import your 'posts' example files (with my changes for the ContainsQueryableValues error) , and it still doesn't work (I mean, no results are found)... so, wait... could be an issue related to the multilanguage/multisite maybe?
I simply followed these steps:
https://statamic.dev/multi-site

[duncanmcclean]

Hmm, it's possible that might be the issue. Are you able to share the project (or an example site with the same search code) and I can take a quick look?

[ivang76]

I think the quicker way would be to install this: https://statamic.com/starter-kits/statamic/multisimplicity
and import your searchable example: https://gist.github.com/duncanmcclean/2f36464ffbe0598a58f40d3cebe5d4e6

because mine contains a lot of crappy tests and experiments and some private data related to our users

[ivang76]

Okay, I can confirm that the issue is definitely related to the multisite/multilanguage setup. In the config/statamic/sites.php, I removed the second language and restored the 'default' key. Now, the search works.
(However, I had to add the Augmentable elements as well, or else an exception blocks the execution: Searchable App\Models\Feed must implement Statamic\Contracts\Data\Augmentable).

So, what's the next step? Is this a bug?

[duncanmcclean]

Hey! 👋

I've just spun up a quick site using the Multisimplicity starter kit for testing and I've managed to get search working with both Runway's built-in search stuff & using custom searchables.

I've pushed up the two implementations on different branches so you can compare against them: https://github.com/duncanmcclean/statamic-cms-9185/branches

I didn't have any issues implementing search with Runway's built-in searchable, everything worked as per the docs.

However, when using a custom searchable, I did have to add sites="*" to the {{ search:results }} tag for any results to be returned. I think Runway does this to workaround this issue in its searchable:

https://github.com/duncanmcclean/runway/blob/5.x/src/Search/Searchable.php#L39-L41

When you're using a custom searchable, you will need to implement both the Augmentable and ContainsQueryableValues contracts. That's not a bug though, we just need to mention it in our documentation.

Hopefully that's helpful for you and you're able to finally get search results returned! 🚀

[ivang76]

Thank you so much, indeed that site="*" (you have a typo, it is site not sites 😉 ) finally get search results returned even with multilanguage/multisite configuration! 👍
And yes, should be useful to mention that missing notes in the documentation, even if now this long git discussion will be a good reference for future users for sure!

Just one final question... now I wanna use a normal laravel route with controller and laravel blade. Which is the right way to query the Search facade to replicate that antlers queries?
with something like this I don't know how to filter on the "site" (or language in my case) if needed:

// Route::statamic('search', 'search');
Route::get('search', [SearchController::class, 'search']);

use Illuminate\Http\Request;
use Statamic\Facades\Search;

class SearchController extends Controller
{
    public function search(Request $request)
    {

        $results1 = Search::index('default')->search($request->input('q'))->get();
        $results2 = Search::index('feeds')->search($request->input('q'))->get();
        $results3 = Search::index('feeds_runway')->search($request->input('q'))->get();
        $results4 = Search::index('posts')->search($request->input('q'))->get();

        return view('search.search', ['results1' => $results1,'results2' => $results2,'results3' => $results3,'results4' => $results4]);
    }

}

@foreach($results1 as $result)
  @if($result)
    <a href="{{ $result->url }}">
        <h2>{{ $result->name }}</h2>
    </a>
  @endif
@endforeach

thanks!

[ryanmitchell]

I would just use Statamic::tag() - and get the search results using the antlers tag in the background.

[ivang76]

thanks, but I need to utilize Blade by passing through a regular Laravel controller.

[duncanmcclean]

If you're unable to use the tag, you can query with the Search facade like you're already doing. Here's pretty much what the tag is doing:

Search::index("database_posts")
  ->ensureExists()
  ->search($query)
  ->withData(true)
  ->where("status", "published")
  ->where('site', '...')
  ->get();

You don't need the ->where('site', '...') condition in places you were doing site="*" in the tag (* causes an early return in the tag).