How to push data via API to Databox
In this document, we'll outline how to sync custom data regularly into Databox from a Cloud-Based application with API access. Databox offers Java, PHP, Ruby, Node.js, GO and PythonSDKs.
-
You can also sync custom data using the Google Sheets integration. Learn morehere.
IN THIS ARTICLE
General Guidelines
There are a few general guidelines to keep in mind while syncing custom data:
- Maximum of 10 requests per second
- Maximum of 500kb in size
- Sync frequency should not be less than every 5 minutes to ensure optimal performance
What to define before syncing custom data
First, we must identify the Data Source where we want to pull our data from, the SDK where we want to push our data to, and a scheduler application so that the data can automatically be synced with Databox on a regular basis.
For our example, we have defined the following:
数据源: Sentry
SDK: Databox SDK
Scheduler: Heroku (free)
如何准备自定义数据
Sentry is an open-source error tracking platform that provides complete app logic, deep context, and visibility across the entire stack in real time. At Databox, we use Sentry to track our backend errors and events. All events are organized in projects.
Our goal is to get create a Metric in Databox to track Events per Project.
Project Name | Today | Yesterday | This Week | This Month |
Project 1 | 10 | 5 | 30 | 200 |
Project 2 | 1 | 30 | 70 | 150 |
Next, we'll look atSentry's API Documentationto determine what we need in Sentry. According to this documentation, we will need the following:
- Authentication token (https://docs.sentry.io/api/auth/)
- List of Event counts for the organization (https://docs.sentry.io/api/organizations/get-organization-stats/)
- List of Projects in the organization (https://docs.sentry.io/api/organizations/get-organization-projects/)
How to sync custom data
Now that our data is prepared in Sentry, we need to sync the data with our Databox Account. Based on thebeplay体育appiosDataBox开发人员文档, we will need the following in Databox:
- 数据源Token (https://developers.beplay体育appioswww.yu-nung.com/create-token/)
- Databox SDK (https://developers.www.yu-nung.com/sdks)
可以在此处找到用于拉动和推动数据的代码http://github.com/databox/sentry2databox.
Pro Tip: When creating a new Custom Token, be sure to manually set up the timezone. By default, the timezone will be set to UTC 00:00.
How to prepare a Scheduler
Now we need to schedule regular data syncs so the data is automatically synced with Databox on a daily basis.Log into your Heroku Account, or create one for free.
学习如何将代码复制到READM HerokuE section ofthis document. You will need to use Heroku CLI to upload your code to Heroku. If you don't already have this in place, usethis documentfor installation steps.
Within your Heroku Account, use the following prompts to create a new SSH public key.
$ heroku login $ heroku git:remote -a sentry2databox $ git push heroku master
When the command completes, you should see a message verifying the deploy.
How to sync data with a Scheduler
Go to theOverviewpage and clickAdd a New Addon. You will need to enter a credit card to add addons, but you will have enough free credits per month that syncing this data with Databox will not create any charges.
Search forHeroku Schedulerand clickProvision.
Click on theSchedulerandAdd a new job.
The last step is to setup theconfiguration variables. Navigate toSettings>Config varsand enter your SENTRY_TOKEN, SENTRY_ORG, DATABOX_TOKEN and LAST_X_DAYS.
You can verify that everything works correctly by selectingMore >Run consolein Heroku and running the job.
If everything is set up correctly, you will see the following:
Additional Information
- Each Rest API Custom Metric supports only one data Aggregation function. To view the same Rest API Custom Metric with different Aggregation functions selected (i.e., "SUM" on one vs "AVG" on another) duplicate Custom Metrics and create the views you desire by followingthese steps.