Performance stats #17
Replies: 3 comments
-
|
Just tested some scenarios using the Step one: Populate the databasecreate loupe database and populate with data (1) ->withSearchableAttributes() default, i.e. everything is indexed(2) ->withSearchableAttributes(['title', 'overview'])(3) like (1) but only with a subset of the data (first 1000 records of
|
Beta Was this translation helpful? Give feedback.
-
|
I did some more tests and research. IndexingOf course, this very much depends on the amount of data. If you index the entire 32k movies from the And you should always configure the SearchingAgain, this also very much depends on the way you're searching and whether typo tolerance is enabled or not. First of all: I've found a few performance bugs in both, Loupe as well as my State Set Index library. So make sure you checkout the Benchmark 1Searching 32k movies for $searchParameters = SearchParameters::create()
->withQuery('nemo')
;Processing time: 85 ms Benchmark 2Searching 32k movies for $searchParameters = SearchParameters::create()
->withQuery('nemo')
->withAttributesToSearchOn(['title'])
;Processing time: 36 ms Benchmark 3Okay, let's see what typo tolerance does. It must slow down the query considerably. Let's say we have a typo in $searchParameters = SearchParameters::create()
->withQuery('mysterios')
;Processing time: 1820 ms I consider this to be amazingly fast! It's a super broad search on 32k movies with just PHP and SQLite 🤯 Benchmark 4Let's do the same as benchmark 3 but only have it search the $searchParameters = SearchParameters::create()
->withQuery('mysterios')
->withAttributesToSearchOn(['title'])
;Processing time: 1476 ms Even better! Remember: It's still searching 32k movies with typo tolerance enabled! This is probably not even a real world scenario. You'd likely filter for other attributes and narrow down the 32k movies in the first place. ConclusionWhen it comes to indexing: Loupe is slow. You should calculate with about 35 documents per second. If you want to give it a try to make it faster, please do! In terms of searching, I consider Loupe to be amazingly fast! I'm over the moon with the results for this early version. Of course, I did lots of performance testing before I published it so I ironed out a lot of mistakes but if you'd asked me at the beginning of the year whether writing a typo tolerant search engine with just PHP and SQLite that searches 32k documents in under 2 seconds was possible, I would've sure said "no way" 🥳 And I'm fairly sure we can optimize all sorts of things and make it even faster out of the box 😊 |
Beta Was this translation helpful? Give feedback.
-
|
I did some more performance tests and optimizations and v0.2 will be able to search the 32k movies in under 400ms 🥳 I've added this to the README in fc6f35b and converted this issue to a GitHub discussion. |
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
-
People have asked me for performance stats which I cannot provide at the moment because I simply don't have any data yet :-)
If you're up to it, feel free to take on this task and run some tests. As a dataset, why not re-using what MeiliSearch provides? https://www.meilisearch.com/movies.json
You'll notice that indexing will take a loooong time but searching should hopefully be pretty quick - of course not MeiliSearch level but quick :)
Beta Was this translation helpful? Give feedback.
All reactions