The Problem

At the time of writing, I have over 17,000 observations across over 6,500 taxa, and a lot more to add. That represents a heck of a lot of scrolling on the respective index pages, when you might only be interested in a very small subset of the data.

The Solution

My checklist functionality resolves this to some extent because, if, for example, you are interested in butterflies, I already have a checklist for those, which shows all my taxa in the Papilionoidea superfamily, nicely arranged and grouped taxonomically. However, I have checklists defined for only six taxonomic groups (I must create some more).

In the meantime, and useful in its own right, I have provided filters on the Taxa and Observations index pages. For taxa, the filter is based only on taxonomy, allowing a root taxon to be selected. Sorting can be based on the most recently added taxa or taxonomically. For Observations, the root taxon filter is also used, along with a filter on observation date. Sort can be based on observation date, date added or date last modified, and can be in ascending or descending order.

In addition, the filters take note of query parameters, to allow the page to be prefiltered and links of such to be shared. The filters use Livewire to allow the page contents to reflect the filter, as soon as a filter option is changed. I will explain how it works for the Observations filter. The Taxa filter follows the same principle but is slightly simpler.

The FilterObservations component

class FilterObservations extends Component
{
    #[Url(history: true)]
    public string $dateFrom;
    #[Url(history: true)]
    public string $dateTo;
    #[Url(history: true)]
    public ?int $rootTaxon;
    #[Url(history: true)]
    public string $sortBy;
    #[Url(history: true)]
    public string $sortOrder;
    public string $rootTaxonName = '';

    public function render(ObservationQueryBuilder $queryBuilder)
    {
        $filterChanged = false;
        if (empty($this->rootTaxon)) {
            $this->rootTaxon = 0;
            $this->rootTaxonName = '';
        } else {
            $this->rootTaxonName = Taxon::find($this->rootTaxon)->scientific_name;
        }
        if ($queryBuilder->rootTaxon != $this->rootTaxon) {
            $queryBuilder->rootTaxon = $this->rootTaxon;
            $filterChanged = true;
        }
        if (empty($this->dateFrom)) {
            $this->dateFrom = '';
        }
        if ($queryBuilder->dateFrom != $this->dateFrom) {
            $queryBuilder->dateFrom = $this->dateFrom;
            $filterChanged = true;
        }
        if (empty($this->dateTo)) {
            $this->dateTo = '';
        }
        if ($queryBuilder->dateTo != $this->dateTo) {
            $queryBuilder->dateTo = $this->dateTo;
            $filterChanged = true;
        }
        if (empty($this->sortBy)) {
            $this->sortBy = $queryBuilder->defaultSort['by'];
        }
        if ($queryBuilder->sortBy != $this->sortBy) {
            $queryBuilder->sortBy = $this->sortBy;
            $filterChanged = true;
        }
        if (empty($this->sortOrder)) {
            $this->sortOrder = $queryBuilder->defaultSort['order'];
        }
        if ($queryBuilder->sortOrder != $this->sortOrder) {
            $queryBuilder->sortOrder = $this->sortOrder;
            $filterChanged = true;
        }
        if ($filterChanged) {
            $query = $queryBuilder->buildQuery();
            $this->dispatch('filterChanged', sqlStatement: $query->toSql(), bindings: $query->getBindings());
        }

        return view('livewire.filter-observations', ['defaultSort' => $queryBuilder->defaultSort]);
    }

    public function clear()
    {
        $this->reset();
    }
}

The properties of the component are simply both ends of a date range, the root taxon and the two parts of the sort (what to sort by and in what order). The important thing to note, is the use of the #[Url(history: true)] Livewire attribute, which adds each property and its value to the URL query string, and ensures that changes in the filter properties are recorded in the browser history, allowing users to go backwards and forwards between different filtered sets. The rootTaxonName property is simply to show the name under the taxon filter field, when the taxon picker is not open.

Another important point is that the FilterObservations component doesn't actually perform any filtering. Instead, if any filter value has changed, the ObservationQueryBuilder helper class builds a SQL query that is dispatched via a filterChanged event. That event is handled by the LoadMore component, which is explained in another article. I will explain the ObservationQueryBuilder helper class after showing the Blade for the FilterObservations component.

<div>
    <div
        x-data="{
            dateFrom: $wire.entangle('dateFrom').live,
            dateTo: $wire.entangle('dateTo').live,
            rootTaxon: $wire.entangle('rootTaxon').live,
            rootTaxonName: $wire.entangle('rootTaxonName').live,
            sortBy: $wire.entangle('sortBy').live,
            sortOrder: $wire.entangle('sortOrder').live,
            filterOpen: false,
            dateFilterOpen: false,
            rootFilterOpen: false,
            sortOpen: false,
            treeViewOpen: false, // Used by the nested TreeView component
        }"
        x-init = "filterOpen = rootTaxon || dateFrom || dateTo";
        x-cloak
        x-on:date-from-changed.window="dateFrom = $event.detail.dateFrom; dateFilterOpen=false;"
        x-on:date-to-changed.window="dateTo = $event.detail.dateTo; dateFilterOpen=false;"
        x-on:root-taxon-changed.window="rootTaxon = $event.detail.selection; setTaxonName(); rootFilterOpen=false;"
    >
        <div class="flex justify-between items-center">
            <h1 class="font-semibold text-xl text-gray-800 leading-tight">
            {{ __('Observations') }}
            </h1>
            <div x-on:click="filterOpen = ! filterOpen" type="button" class="inline-flex items-center" title="{{ __('Show/Hide Filter/Sort') }}">
                <x-icons.filter class="cursor-pointer text-gray-400 hover:text-gray-500" width="16" />
            </div>
        </div>
        <div x-show="filterOpen" class="relative mt-2 border-t pt-3">
            <div class="flex flex-col sm:flex-row space-y-3 sm:space-x-3 sm:items-baseline">
                <div class="text-ts font-bold">{{ __('Filter') }}</div>
                <div>
                    <x-dropdown-button toggle="dateFilterOpen" label="Observation Date" />
                    <div x-show.transition="dateFilterOpen" x-on:keydown.escape="dateFilterOpen = false" x-on:click.away="dateFilterOpen = false">
                        <x-date-range-picker :start-date="$dateFrom" :end-date="$dateTo" />
                    </div>
                    <div x-show.transition="!dateFilterOpen && (dateFrom || dateTo)" class="text-xs">
                        <p x-text="dateFrom + ' - ' + dateTo"></p>
                    </div>
                </div>
                <div class="relative">
                    <x-dropdown-button toggle="rootFilterOpen" label="Root Taxon" />
                    <div x-show.transition="rootFilterOpen" x-on:keydown.escape="rootFilterOpen = false" x-on:click.away="rootFilterOpen = false" class="absolute left-0 top-10 z-40 w-full sm:w-120">
                        <input type="hidden" x-ref="rootTaxon" x-model="rootTaxon" />
                        <x-root-taxon-picker />
                    </div>
                    <div x-show.transition="!rootFilterOpen && rootTaxon" class="text-xs">
                        <p x-text="rootTaxonName"></p>
                    </div>
                </div>
                <div>
                    <x-dropdown-button toggle="sortOpen" label="Sort" />
                    <div x-show.transition="sortOpen" x-on:keydown.escape="sortOpen = false" x-on:click.away="sortOpen = false">
                        <div class="border rounded-md mt-3">
                            <select x-model="sortBy" class="focus:outline-none border-0 py-1 rounded-l-md">
                                <option value="Time Observed">Time Observed</option>
                                <option value="Time Recorded">Time Recorded</option>
                                <option value="Time Updated">Time Updated</option>
                            </select>
                            <select x-model="sortOrder" class="focus:outline-none border-0 py-1 rounded-r-md bg-gray-200">
                                <option value="Ascending">Ascending</option>
                                <option value="Descending">Descending</option>
                            </select>
                        </div>
                    </div>
                    <div x-show.transition="!sortOpen" class="text-xs">
                        <p x-text="sortBy + ' ' + sortOrder"></p>
                    </div>
                </div>
            </div>
            <div class="absolute top-3 right-0">
                <x-button x-on:click="window.location.href = window.location.pathname;">Clear Filter</x-button>
            </div>
        </div>
    </div>
</div>

The filter contains three parts:

• a date range picker (using my own component)
• a tree view component, explained in another article
• a control with two select elements, for the sort field and order, respectively

All controls have text underneath to display the chosen option, when the control is collapsed.

Initially the filter is hidden, and is toggled by the value of filterOpen. If the URL contains query parameters for either of the filters, the filter is opened in x-init.

The date-from-changed and date-to-changed events are dispatched by the date range picker component. The root-taxon-changed event is dispatched by the tree view component. It uses a compound key for each taxon, based on the component used and the taxon ID. Here we are only interested in the taxon ID, so we extract that to set the rootTaxon.

The Clear Filter button simply refreshes the page using window.location.href = window.location.pathname;. I tried using the reset() method of the Livewire component but it had the effect of writing 5 entries to the browser history, one for the clearing of each property of the filter.

The ObservationQueryBuilder helper class

The ObservationQueryBuilder helper class is used by the FilterObservations component to build an Eloquent query, from which the SQL statement and bindings are passed to the LoadMore component to fetch the models that match the filter properties. It is also used by the index method of the ObservationController to ensure that, when the page is first loaded, any existing query parameters are taken into account to filter/sort the results. ObservationQueryBuilder is included in the via dependency injection and defined as a singleton in AppServiceProvider. This is to ensure that its properties are maintained between its use in the ObservationController and FilterObservations component, avoiding an unnecessary refresh of the observations displayed on the page.

class ObservationQueryBuilder
{
    public string $dateFrom;
    public string $dateTo;
    public ?int $rootTaxon;
    public string $sortBy;
    public string $sortOrder;

    const SORT_OPTIONS = [
        'Time Observed' => 'observed_at',
        'Time Recorded' => 'added_to_inat_at',
        'Time Updated' => 'updated_in_inat_at'
    ];
    const SORT_ORDERS = ['Ascending' => 'ASC', 'Descending' => 'DESC'];
    public array $defaultSort = ['by' => 'Time Observed', 'order' => 'Descending'];

    public function __construct()
    {
        $this->dateFrom = '';
        $this->dateTo = '';
        $this->rootTaxon = 0;
        $this->sortBy = '';
        $this->sortOrder = '';
    }

    public function buildQuery()
    {
        $query = Observation::where('id', '>', 0);
        if (!empty($this->rootTaxon)) {
            $rootTaxon = Taxon::find($this->rootTaxon);
            $rootTaxonDescendantIds = array_values($rootTaxon->descendants->pluck('id')->toArray());
            $rootTaxonDescendantIds[] = $rootTaxon->id;
            $query->whereIn('taxon_id', $rootTaxonDescendantIds);
        }
        if ( ! empty($this->dateFrom) ) {
            $timeFrom = $this->dateFrom . ' 00:00:00';
        }
        if ( ! empty($this->dateTo) ) {
            $timeTo = $this->dateTo . ' 23:59:59';
        }
        if ( !empty($timeTo) && !empty($timeFrom) ) {
            $query->whereBetween('observed_at', [ $timeFrom, $timeTo ]);
        } elseif ( !empty($timeFrom) ) {
            $query->where('observed_at', '>=', $timeFrom );
        } elseif ( !empty($timeTo) ) {
            $query->where('observed_at', '<=', $timeTo );
        }
        $sortBy = empty($this->sortBy) ? 'observed_at' : self::SORT_OPTIONS[$this->sortBy];
        $sortOrder = empty($this->sortOrder) ? 'DESC' : self::SORT_ORDERS[$this->sortOrder];
        $query->orderBy($sortBy, $sortOrder)->orderBy('id', $sortOrder);

        return $query;
    }
}

Basically, all this is doing is dynamically building a query based on the filters. It has only two types of filter: one based on the observation date; and the other on the root taxon. It also has the option to sort by one of the three dates (when the observation took place, when it was added to iNaturalist, and when it was last updated in iNaturalist) and to choose between ascending or descending order. One of the great things about the Builder is that you can gradually and conditionally build up your query as long as you are calling methods that return a Builder (i.e. until you call any method to get the results).

It starts by adding a dummy WHERE clause. This is simply to give us a Builder object, to which we can add the rest of our query. The date filter will normally work with a date range, using Builder->whereBetween() to add a BETWEEN condition to the WHERE clause based on the selected date range. If only one end of the date range is supplied, it uses >= or <=, accordingly. The taxon filter gets an array of the ids of the descendant taxa of the root taxon (picked in the filter) and uses the Builder->whereIn() to add a IN condition to the WHERE clause.

The descendants are retrieved using a method on the HierarchicalModel class, which is inherited by the Taxon model.

public function getDescendantsAttribute()
{
    return static::where('set_start', '>', $this->set_start)
        ->where('set_end', '<', $this->set_end)
        ->orderBy('set_start', 'asc')
        ->get();
}

Note that this uses the Nested Sets Model, which is explained in another article.

The new SQL statement and its bindings are dispatched via a filterChanged event that will be picked up by the any LoadMore and LoadMoreButton components, and the set of observation tiles will be changed accordingly.