Loading speed of "Do a search for" VS "List of Things, Things"

Need help Database
1

Hello everyone. I’m not exactly looking for help, as much as I would like to see what everyone’s experience is on database loading speed using two different methods, so that going forward I’m creating my databases in the most efficient way possible.

Since there is often more than one way to accomplish the same task on Bubble, I’m not sure which is really the “right” way to do it. When I started on the platform, I began building an app that grew and became a monster. The database has a few million entries spanning over 60 data types. The user’s dashboard necessitates that it is a one-page application, so there are many groups that show/hide based on custom states. Many of those have repeating groups to display various data types.

At the beginning, I didn’t really know of any other way to display the data in repeating groups other than using “Do a search for…” and adding the constraints, such as “Clients > Servicing Representative = Current User” to show only the list of clients that pertained to that particular “Representative”.

As time went on, I refined the database structure to save data types against other data types. Such as “Representatives” have a list of “Clients”, and so forth. My though with this was that having a smaller list of potential data to work with for each user would make repeating group load times much faster.

When the application contains thousands of “Clients” and you have to perform a search for only those “Clients” that match the particular constraints, it seems as though it would be much quicker to only display the few hundred “Clients” that belong to a particular user.

So, I set an example repeating group to display “Current User’s List of Clients” rather than “Do a Search for Clients > Representative = Current User”.

Now, here is where the questions comes in, and I apologize for the lengthy set up. What is you experience with loading times in your groups using these different methods. I actually found that the “Display list of Current User’s Thing” method was actually far slower than performing a search for a “Thing” with the necessary constraints. And this is a difference between less than a second to perform a search and display, vs 8-10 seconds of loading time to display a list of users “Thing” in the group. What is your experience, and is there something that I’m missing? Thank you for your input and feedback. Like all of you, I’m just trying to decide on the best way to serve information as quickly as possible to the user, when you have a big database.

Example:
This method is very fast
Repeating Group: Type = “Client”
Data Source = “Do a Search For Clients”
Constraint = “Representative = Current User”
Sort by: Last Name

VS

This method is very slow
Repeating Group: Type = “Client”
Data Source = “Current User’s Clients”: Sorted by Last Name

11 Likes
2

It does seem a bit counter-intuitive, but my experience has also been to do method 1 listed above for faster results. From previous forum posts, I believe this is because the search runs server side (much faster) when using this method instead of client side.

What I do now currently in the app I’m making is run a couple default searches on page load, which get stored in states. Throughout the app, I’ll then manipulate these searches by using the :intersect with [new search] :unique elements. Seems to be working pretty fast for repeating groups, but I don’t have nearly as much data to handle as you do, and would be curious on if there’s a better approach to this.

5 Likes
3

Thanks @callen.hedglen. After doing some further testing, I did decide to revert back to the search method. I think I understand what you are getting at that it’s quicker because the search is happening on server side, vs the list being pulled client side. That makes more sense. I appreciate your feedback.

4

Sure thing! To add onto this, also make sure to do any sorting inside of the “Do a search for…”. If you add it on afterwards with :sort by, that will make it run on the client side and be much slower.

1 Like
5

@jcindy81 How fast is the ‘Do a search for’ with millions of records?

Also about your one page app. Do you use display data in a repeating group workflows or are you using ‘Do a search with and you get constraints from staits or parent group with your hidden groups? Any speed difference with this?

1 Like
6

Even with a huge database, the do a search for returns results immediately. At, least there is no noticeable wait time.

I have each repeating group set with “do a search for parent groups thing” and the appropriate constraints. Then all the various filter options for the data also use the same function, with “ignore empty constraints” checked to bypass any search filters that were left empty. The data still displays immediately even as the number of constraints grows. @callen.hedglen is also correct that any sort functions should be performed during the search portion, as sorting after the search adds a lot of time to displaying results.

I have not noticed any difference in using “search for a thing” as opposed to “search for parent groups thing”.

What I’m beginning to encounter now is my editor slowing down because of the amount of elements and groups that my one page app is using. I can work in it for about an hour before the editor crashes and needs to reload. I’m not sure if this is a bubble memory limitation, or my hardware. My machine has a core i3 and 24GB of RAM, so I don’t think it’s a memory issue on my end, but I don’t know enough about the bubble side to be sure. I also don’t keep extra tabs open when I’m in the editor, to keep browser memory open. Still, the editor is pretty sluggish to save after each change I make.

5 Likes
7

Hi

When the editor starts slowing down, just hit ctrl f5.
(Chrome). This will reset the cache and get your editor going
For the next hour.

6 Likes
8

Hi @jcindy81…
Thanks for starting a great post!
Couple of quick thoughts…

  1. I’ve heard Firefox handles the memory better for large designs.
  2. I also have one huge page - about 40 instances of 40 different reusable elements each with about 60 elements on them. Takes a while to load, but still edits well.
  3. I’ve moved from lists of items to searches too. Much easier to handle and less to write when new things are created. What takes the time with searches is the amount of things returned rather than the amount to be searched through.

Good luck!
Antony. :slight_smile:

9

Very useful !! Thanks guys :wink:

10

Question guys…

Since “do a search for” is server side, and “list of things, things” is client side, “do a search for” will tend to be faster but will contribute to an increase in server capacity. Is that right?

Also, for a use of FULL CALENDAR.
What’s better, “Do a search for” or “List of things”? Will it hugely affect server capacity since we are talking about calendar?

1 Like
11

Hi @callen.hedglen !

I am coming a bit late but…
Could you please elaborate more on that? What would you suggest to perform a fast sorting in a similar situation?

Thanks,
Bye!

Carlo

12

I have hands on experience with the search functionality. There are two use-cases to be aware of as mentioned previously.

Let’s say you have a User with data records related to Invoices, in the early stadium linking to the Current User’s related Invoices will be extremely fast, now wait until that User has generated more than 500 invoices. In this case, you definitely DO NOT want to have that record Invoices as a list inside the User table.

Best case here is to use Do a search for Invoices, where the User is Current User and not linking the Invoice record at all on the User’s table. Instead record the User on the Invoice Table as a link.

Second case would be where the User has some records related to their own personal information or subscription details. Best option here is to link that record inside the User table and referencing the Current User’s Subscription.

This is the way to go, there has been a lot of discussion on this but I recommend everyone to follow the use-cases as outlined above in my reply.

Good luck all & Happy Bubbling.

FYI, I had a data record of 9,000 invoices as a record inside my User table, and it was slowing everything down, even looking for the User’s subscription information was extremely slow due to the Invoices records inside the User table.

3 Likes
13

This topic is the #1 thing that paralyzes me when trying to build. Not knowing which way to structure data for scalable performance for various situations. Anyone provide coaching on just this subject?

The census on this thread is use “Do a search for” data types that have reference links to other data types instead of Lists of things. In what situation will a List of Things yield the better performance?

Example:

  • I have a marketplace with 4 user types… they all need profiles… there’s calendars, events, transactions, reviews, messages, and tons of other data types required.
  • My users have profiles… but profiles can have a list of users.
  • Profiles can have Transactions, Reviews, Calendar Events, Favorites, ect. (Should those be lists of things or should I link the Profile to each separate datatype.)
14

Here is a discussion many of us would love to know more about.
Would Bubble officialy give us a tutorial, “Loading Data Time - Best Practices For Each Process”, @eve ?

3 Likes
15

So, in my case I need to register how many exercises my Users have done correctly… Every time they enter in the app, it needs to be counted.

Currently, my app is recording it inside the User (some users have 2000+ records in 2 years of use), in this way my database will not grow bigger with the time. (let’s say a database of 100 items for every active user)

On the other hand, if I don’t register it inside the user, the same database would grow bigger, (let’s say a database of 2k items each user in history)

So, in short, what’s better, to keep 2k items inside the user, or to search 2k items inside a 200k items database?
Do you have any thoughts about if I should keep this way, or if I should change?

16

Do a search :slight_smile:. Bubble uses Postgres and searches should be extremely fast.

1 Like
17

So even if generates a database with over a million items? wow, that’s nice to know haha it changes a lot in the way of programming

18

my question is, for nested repeating groups

typically you dont want " do a search for " in each cell of a repeating group, so my app is structured as

users have jobs
jobs have products

first repeating group, do a search for jobs
repeating group inside of each cells job is “cur cells jobs product list”

problem is i dont know whats happening for this to occure. is all product data being searched for when the jobs are being searched for?

when and how is it opening up the jobs list of products to fill in the nested RG of products?

because its definatly slow to “do a search for” products that ref cur cells job.