Skip to content

Querybook 2022 Update

Jump to bottom
J.C. Zhong edited this page Jan 5, 2023 · 2 revisions

Following are the top new features we have added during the year 2022:

  • Query Search: Query executions are searchable now.
  • UDF Creator: Custom UDF plugin can be added to let users add UDFs easily in the query editor.
  • Schedules Page: Users can see all their scheduled DataDocs in a central place now.
  • Dynamic Result Size: Dynamically configure how many rows of results to display on the UI.
  • Table Uploader: Create tables from query execution results or file uploads.
  • Customized Tags: Annotate tags with features like color, tooltip, icon, rank and etc.
  • List V2: Added nested lists, full page view, and more collectible types.
  • Automatic Query Limit: A limit clause will be added to a select query without limits.
  • Live Syntax Check: Validate query syntax and show errors in realtime.
  • Query Transpiler: Transpile queries between different languages like Presto and SparkSQL.
  • Dag Exporter: Export a DataDoc as a DAG to systems like Airflow.
  • Customized Schedule Notification: Scheduled Docs can send notifications to custom Slack alias or email addresses.
  • Event Logging: Provides instrumentation support for logging client action events and API events.
  • Run All Cells: Run all query cells in a DataDoc.

Feature highlights

Query Search

Users can search query executions by statement type, query engine, and runner. Can also put multiple tables to see how they are used together.

querysearch

UDF Creator

If your query engine supports UDFs/Stored Procedures, you can use the custom UDF plugin to let users add UDFs easily in the query editor. The UDF editor in Querybook allows users to quickly create UDFs by filling out a form.

image image

Schedules Page

We added a new page to display all your data docs.

scheduled1

By default, it only shows the scheduled docs, from where you can view and manage the schedules and their run history.

scheduled2

You can also filter or view all the data docs you own.

scheduled3

Dynamic Result Size

Users can now dynamically configure how many rows of results they want to see on the frontend.

resultpreview1

They can also apply this limit for charts, in which case if it is higher than the default limit, only the chart data would be fetched.

resultpreview2

Table Uploader

Users can create and upload a table from a query execution result or a file.

Upload from query execution

tableupload1

Upload from sidebar

tableupload2

Select source (can be query execution as well)

tableupload3

Fill out table details

tableupload4

Upload

tableupload5

Customized Tags

Added ability to annotate tags with the following features:

  • Color
  • Tooltip
  • Icon
  • Rank
  • Admin (Only admin can edit)

tag1

You can edit tags by right-clicking (only the ones you are allowed to)

tag2 tag3

List V2

Updated the List feature with

  • Full page view
  • Nested lists
  • Public lists
  • Add custom descriptions to each list item
  • Share ith R/W permissions
listv2

Automatic Query Limit

If a select query doesn't have a LIMIT clause, Querybook will add a default one for it automatically. Querybook will also tell users if the number of rows returned has reached the limit. Users can turn it off by selecting the none option.

querylimit

Live Syntax Check

Check the query syntax and show the errors in realtime. The only default support right now is Presto/Trino.

syntax

Query Transpiler (Experimental)

Users can now transpile queries between Presto and SparkSQL.

transpile transpile2

Dag Exporter (Experimental)

You can add a dag exporter plugin for your dag system to allow users to export a DataDoc as a dag to your dag system.

dag1 dag2 dag3

Customized Schedule Notification

  • Users can add a separate notification section for each notify type in the schedule form
  • For each notify type, it will have a "Notify To", where users can put either multiple email addresses or slack channels/ usernames
  • The scheduled datadoc will be always run by the datadoc owner

notification1 notification2

Event Logging (Experimental)

Added support for event logging with some built-in loggers. Admins can add other loggers through the plugin system.

DBEventLogger output example

eventlogging

Run All Cells

Users can now run all query cells of a DataDoc in sequence, like the manual run of a schedule.

runall

Small Feature Improvements/Bug Fixes

  • Support embedding of DataDocs in external Applications, like Linkedin Datahub
  • Replaced D3 with React-flow
  • Manually refresh a table from metastore
  • Added column filtering to toggle which columns to show
  • Template variables support for ad-hoc query
  • Sync table/column descriptions from metastore
  • Added icon for table partition keys
  • Show deactivated user with different UI
  • Added syntax highlight/copy to markdown code
  • Syntax highlighting for templated queries
  • Added schema filtering for table search
  • Added task to auto-disable unused scheduled docs
  • Ability to sort templated variables
  • Warnings when dropping tables
  • Search table with their exact name would auto open the table
  • Autosuggest would now add quotes around columns and tables when needed
Clone this wiki locally