Schedule API workflow on a list - does the list have a limit?

How do you break up the results of a search into a list of shorter lists?

Like, I’m trying to delete +100K records and of course the action “delete a list of things” fails. So maybe I can schedule an API workflow on a list, where each item in the list is a list of 500 records? Then the API workflow would delete each shorter list of records.

1 Like

Could you do it by date created or are created dates all more or less the same?

I’m not sure how we’d get a list of lists in either case. Can you describe how it might work?

Looking at the list expression options, there isn’t even a way to get item-X-to-item-Y. The only options are

  • first item
  • last item
  • a random item
  • a specific item
  • first item to item-X
  • item-X to last item

So I can’t even do something like “500 random items”, let alone “item 300 to item 400”. If I could do repeats I could take the first 100 items, then delete the first 100 items, then repeat, but I can’t do that. If I could reference the other cells of a repeating group I could have one cell take the first 100 items, then the next cell could run off a list that filters out the first 100 items, taking the first 100 items of that list, and so on, but I can’t do that either.

2 Likes

When you select the ‘list to delete’ in the API workflow, do you do a search for a Thing? As a constraint, the date is there. I have them setup where I set created date < current date changed in X and date > current date changed in Y.

What I dont know for sure is if you need to create new endpoints for each list or if you can have them as a new workflow action.

Okay, but how do you do that more than once? I see how you could get one list of records that fall between two date/times. How do you get a list of lists?

So you want to delete a list of Things that is part of another Thing?

Could you do do a search for Things Thing:filtered > advanced where This Things creation date etc etc? Seems possible but havent tried though :smiley:

Is it true that the list to run a work flow on cannot have only one item in it? I am scheduling a workflow on a list that varies in size depending on when its called (its an API) and it seems when I call it and there is only one item returned in the API then the scheduled workflow (which saves what is returned in the API as things) does not work. When i change the parameters of the API to include more than one item, then the workflow does run.

@josh - any update on this? Thanks

2 Likes

Did you find a solution?

1 Like

No. As far as I know the “schedule API workflow on a list” function is still limited and unreliable.
I went and got a whole different server to do analysis and processing on large batches of data and I might replace all of the “schedule API workflow on a list” actions with calls to the other server just because I don’t fully trust this function to not make any mistakes.

What server are you using now for the analysis?

hostm.com
They’re awesome. I don’t have much experience picking hosts, but their customer service has always been impressive and I like their straight forward one-price-no-limits deal.

I’m using the API to pull data to the hostm server where I can do whatever I want with it. I looked at turnkey databases and dashboards but they all wanted a direct connection to the bubble database, which I can’t get because I’m not on a dedicated hosting plan.

I haven’t used logic on the hostm server to manage actions in bubble yet, but it should be do able.

How would you break it into sets of 100?

2 Likes

Hi, coming into this train late. I know the Bubble team is currently working at it (Bubble’s Roadmap).

Is there a good work-around for this? What is a good way to do batches of 100s?

I stubbornly wasted quite some time trying to get around this instead of asking for help. When I finally asked for help the solution was clear within minutes.

I’m writing this as a:

  1. note-to-self (I tend to forget),
  2. way of helping someone else in the future; and,
  3. getting some feedback from the Bubble hive-mind.

Here is the solution I came up.
tl;dr learn how to use recursive workflows.

Some context: I need to:

  1. create a list of Apartments within Buildings (100-300 apartments per building, up to 30 buildings), and
  2. add those created Apartments to the Building’s List of Apartments.

It’s a poor-man’s graph database, I guess. The problem was that I would schedule an API workflow on a list and, because of well-documented limitations of Bubble’s database, these items would either not be created at all or they would not be added to the Building’s List of Apartments.

The main problem was that it was trying to add multiple Apartments to one Building’s List of Apartments. That lead to data inconsistencies.


The solution I came up with - with the tutelage of the all-knowing :woman_technologist: @romanmg :woman_mage: - was to create a recursive workflow for each Building. So that only 1 Apartment is added to the Building per iteration.

When the user clicks a button, it schedules an API workflow on a list; running on a list of Buildings and I pass a list with texts that will be used to create the Apartments.

  1. that API Workflow creates an Apartment
  2. then adds it to the Building
  3. schedules itself again, but this time it takes that list of texts:minus list of texts:first item (which was the item just used).
  4. it will stop scheduling itself when that list of texts:count is 0

This is probably child’s play for you pros, but for me it was like the first time I discovered VLOOKUP in Excel.


I hope this helps some poor soul in the future.

Thanks! :bowing_man:

20 Likes

I couldn’t edit the previous post, but I wanted to improve it:
list of texts:minus list of texts:item #1. Using first item could potentially remove various items.

I learned that from @keith on this post: "minus item: first item..." expected behavior?

2 Likes

Would you be willing to share some screenshots of the workflow you described in this post?

Ooops, I never saw this. Here are the screen shots.

Workflow to create buildings

step 1

step 2

step 3

Here’s the workflow to create apartments.

step 1

step 2

step 3

9 Likes

I think this is the type of solution I am looking for but I really don’t see how can I create the first step which is to schedule the workflow. I can see the “Endpoint” workflow but Idk how to create this.

I was wondering to user this resource to delete a list of entries that is creating a “time out” due to the amount of data, so every time the user wants to delete an item from his list, he will actually “hide” this from his view by setting a “yes” field and all entries checked as “yes” needs to be deleted by using a scheduled workflow.

So, how could I achieve this?

Thanks a lot!!

(let’s ignore the fact that 1 year has elapsed…)
I often have to create 2 Workflows. 1st kicks it off and the 2nd one is the actual recursive workflow.


Could database triggers be more suitable for this use?.

1 Like