Re-harvesting of data

Modified on Mon, 15 May 2023 at 11:03 AM

Sometimes the publisher recalculates data for previous month(s) and makes changes to them. What can we do about this to correct our data in CELUS?

The answer is simple - delete the obsolete data in CELUS and harvest them again. Right now this is only possible in two steps - first delete the data, then harvest them again (don't worry, we already have a "Reharvest solution" in our pipeline.)

The first step is to delete data for specific months manually platform by platform now by going to Platforms and clicking on "OVERVIEW":

The following dialog opens, where you need to activate the Delete mode, select the months for which you want to delete the data and click on "DELETE":

Then you deactivate the Delete mode and harvest the data again (by selecting the months you want to download and clicking on "HARVEST").

Alternatively, you can delete ALL platform data by going to Platforms → the platform you want to delete data for → Data management and clicking on Delete all platform data

However, be VERY careful with this button, because it really deletes all data from the platform, so in case you previously uploaded some data manually, you would have to upload them again. Also if you have historical data from a long time ago, sometimes they are no longer available to harvest via SUSHI. Also notice that in this dialog, you are deleting data for the institution you have selected in the upper left corner. In the case of the screenshot above, All institutions are selected, so by clicking the "Delete all platform data button", we would delete the data from the selected platform for all institutions in your CELUS.

Now let's move to harvesting.

You can harvest data for all institutions in bulk by going to Platforms → the platform you want to harvest → SUSHI. Here you can click on the box next to Title, which will select all of the SUSHI credentials for this platform that are listed below:

Click on Harvest, the following dialog opens, where you can choose the months you want to harvest data for (in case you want to harvest data for only one month, make the start date and end date the same, e.g. 2022-06) and click on the "Start xx downloads" button.

Now the data is being harvested.