New Features and Changes in Release 2.5 of the Splice Machine Database

This topic describes significant new features and changes in the Splice Machine database, which is used in both our Database-as-Service and On-Premise Database products, with notes for each patch release since the initial release of version 2.5, in these sections:

2.5.0.1805 Patch Release 5-Feb-18

JIRA-ID Description
Internal Improve export performance
Internal Fix illegal merge join
Internal Prevent incremental backup from losing changes

2.5.0.1804 Patch Release 28-Jan-18

JIRA-ID Description
Internal Add hint useDefaultRowCount and defaultSelectivityFactor
Internal Fix upsert index
Internal Vacuum disabled tables
Internal Cache more database properties
Internal Fix database restore from s3
Internal Retry write workload on host unreachable exceptions

2.5.0.1803 Patch Release 21-Jan-18

JIRA-ID Description
Internal Add system procedure to enable/disable all column statistics
Internal Close FutureIterator to avoid race condition

2.5.0.1802 Patch Release 14-Jan-18

JIRA-ID Description
Internal Clean up incremental changes from offline regions that were offline before full backup
Internal Add ignore txn cache
Internal Handle CannotCommit exception correctly
Internal Reduce lengthy lineage of transformation for MultiProbeTableScan operation
Internal Add JDBC timeout support
Internal Get keytab file name correctly

2.5.0.1749 Patch Release 26-Dec-17

JIRA-ID Description
Internal Exclude metrics jar for cdh platforms
Internal Add missing getEncodedName() method

2.5.0.1748 Patch Release 18-Dec-17

JIRA-ID Description
Internal Fix inconsistent selectivity with predicate order change
Internal Delete data by region without scanning
Internal Adjust selectivity estimation by excluding skewed default values
Internal Propagate SORT elimination to outer joins
SPLICE-1920 Tooling to fix data dictionary corruption

2.5.0.1747 Patch Release 08-Dec-17

JIRA-ID Description
Internal Terminate backup timely if it has been cancelled
Internal Prepend schemaname to columns in PK conditions in update statement generated by MERGE_DATA_FROM_FILE
Internal Fix column misalignment for join update through mergesort join
Internal Bound the outer join row count by the left outer table's row count
Internal Parameter sanity check for region operations
Internal Populate default value for column of DATE type in defaultRow with the right type (regression fix for SPLICE-1700)
SPLICE-1700 Avoid updating existing physical rows when adding not-null
SPLICE-1802 Create index by HFile loading(2.5)

2.6.1.1745 Patch Release 08-Dec-17

2.6 was an interim release that has been folded in to 2.5

JIRA-ID Description
Internal Add region name and id to compaction job description
Internal Add more logging
Internal Enhance MultiProbeIndexScan to apply for cases where inlist is not the leading index/PK column
Internal Add logs around task failure in Spark
Internal Make table level select privileges to column level
Internal Add back synchronous backup/restore
Internal Add check for actionAllowed
Internal Exclude flatten and restart scripts
Internal Move the bulkimport/bulkdelete test in DefaultIndexIT to a separate file under hbase_sql so it won't run for mem platform
Internal Add parameter to ignore missing transactions from SPLICE_TXN
Internal Keep alive backup
SPLICE-1802 Create index by HFile loading
Internal Synchronize access to shared ArrayList
Internal Fix automatic DROP of temp tables
Internal Enhance subquery processing logic to convert eligible where subqueries
Internal Allow first connection on Kerberized cluster
Internal Large variance in simple select query execution time
SPLICE-1700 Avoid updating existing physical rows when adding not-null column with default value
Internal Allow distributed execution for splice client
Internal Avoid index creation timeout
Internal Flatten correlated Scalar Subquery in Select clause
Internal Null checking for region operations
Internal Avoid sort if some of the index columns are bound with constant
Internal Set rowlocation for index lookup

Splice Patch Release 2.5.0.1735

JIRA ID Description
SPLICE-31 Our JDBC driver now supports Kerberos authentication.
SPLICE-662 Our JDBC client now distributes query requests across multiple nodes without losing transactions, which provides a more even distribution of work across the cluster, resulting in better performance.
SPLICE-1320 Adds ARRAY data type
SPLICE-1372

We've added two new system procedures for working with database operations that are currently running on a server:

SPLICE-1373

We've made two significant changes in logging:

  • Statement logging is now enabled by default. To modify this, see the Using Logging topic.
  • Logs are stored, by default, in your region server's log directory. You can modify the location of the logs with a code snippet that is described in the Installation Guide page for your platform (Cloudera, Hortonworks, or MapR).
SPLICE-1482 Improves import performance by using Hfiles.
SPLICE-1512 Allows use of multiple DISTINCT operations in aggregates.
SPLICE-1591 Adds an option to purge deleted rows when performing a major compaction.
SPLICE-1603 Adds a clause to the ANALYZE TABLE command to allow analyzing via sampled statistics, rather than analyzing the entire table.
SPLICE-1607 You can now configure and use SSL/TLS secured JDBC connections to your Splice Machine database. We support both basic encrypted connections and peer-authenticated connections. Please see our Configuring SSL/TLS topic for more information.
SPLICE-1617 Adds support for suppressing NULLs when creating an index to address a situation where most rows in a table contain the default (NULL) value, which wastes storage and causes bad distribution of rows across regions.
SPLICE-1669 Supports bulk delete via HFiles, bypassing the HBase write path.
SPLICE-1671 Added the ability to take snapshots of tables (and their indexes).
SPLICE-1701 Adds ability to monitor memory usage by HBase JVM through JMX.
SPLICE-1760 The new SYSCS_UTIL.GET_SESSION_INFO" system procedure reports information about your current session, including the hostname and session ID for your current session. You can use this information to correlate your Splice Machine query with a Spark job: the same information is displayed in the Job Id (Job Group) in the Spark console.
Internal

We've added a number of built-in system procedures that allow you to work directly with table and index regions; some customers use these to optimize processing of recent data while optimizing storage of older data:

Internal Imports data without overwriting generated or default values contained in non-imported columns.
Internal Our ODBC Driver now supports BLOB and CLOB objects.

Splice Patch Release 2.5.0.1729 01-Aug-17

JIRA ID Description
Internal add a system procedure MERGE_DATA_FROM_FILE to achieve a limited fashion of merge-into
Internal Implement Kerberos JDBC support (2.5)
SPLICE-1482 HBase Bulk Import
SPLICE-1591 Allow Physical Deletes in a Table
SPLICE-1603 Support sample statistics collection (via Analyze)
SPLICE-1671 Enable snapshot with bulk load procedure

2.5 GA Release 01-Mar-17

JIRA ID Description
Internal Privileges at schema level.
Internal Incremental backup/restore.
Internal Non-native database dump/load.
Internal ORC and Parquet implemented as external tables.
Internal SQL WITH clause.
Internal Window function enhancements.
Internal Caching of RDDs (pinning).

For a full list of JIRA’s for the Community/Open Source software, see https://splice.atlassian.net