Using Attunity to Export MySQL Tables to Splice Machine

Follow these steps to use Attunity Replicate to export tables from a MySQL database and import them into your Splice Machine database:

  1. Create a Splice Machine Endpoint in Attunity
  2. Create an Attunity Task to Export Data
  3. Run the Replication Task
  4. Import the Data into Splice Machine

1. Create a Splice Machine Endpoint in Attunity

Your first step is to open the Attunity Replicate user interface and click Manage Endpoint Connections.

In the pop-up window that opens, select + New Endpoint Connection, and follow these steps:

  1. Enter data in these fields:
    1. Enter the name for your endpoint in the Name: field.
    2. Enter a description for the endpoint in the Description: field.
    3. Select Target as the value of the Role: field.
    4. Enter Hadoop as the value of the Type: field. You'll now see three new fields displayed: Security, HDFS, and Hive Access.
  2. Click the down-arrow to fill in Security information:
    1. Select User name in the Authentication type field.
    2. Enter the Hadoop username in the Username: field. For example: hdfs.
  3. Click the down-arrow to fill in HDFS information:
    1. Select WebHDFS in the Use: field.
    2. Enter the IP or host name of a Hadoop name node in the NameNode: field.

      Make sure all of the data nodes are accessible from Attunity and configure all of the Hadoop nodes in /etc/hosts file in the machine that is running Attunity.

    3. Enter the port to use for webHDFS in the Port: field. The default port value is 50070.

      Make sure the WebHDFS service is running.

    4. Enter your preferred target location in the Target folder: field. Make sure that the user you specified in the Username: field has write access to this folder.</p>
  4. Click the down-arrow to fill in the Hive Access information:
    1. Select No Access in the Access Hive using: field.
  5. Now select the Advanced tab near the top of the window, and click the down-arrow to display the File Format settings:
    1. Change the Field Delimiter: character to |.
    2. Click the Save button at the bottom of the window to save your endpoint settings.
    3. Click the Test Connection button to test your settings.

2. Create an Attunity Task to Export Data

Now that you’ve created the endpoint, you need to create an Attunity Replicate task that exports data from the MySQL source to files that Splice Machine can import.

Start by clicking + New Task in the Attunity Replicate UI, and then follow these steps:

  1. Enter the name for your task in the Name: field.
  2. Enter a description for the task in the Description: field.
  3. Select Unidirectional in the Replication Profile field.
  4. In the Task Options section:
    • Select Full Load.
    • Select Apply Changes.
    • If you want to store the change logs on the target you selected, select Store Changes.
  5. Click OK to save the task settings and proceed to the Table Selection settings.
  6. Drag and drop the source and target endpoints into the center panel. Select the Splice Machine endpoint you just created as the target endpoint.
    1. Click Table Selection to open the pop-up window for selecting tables.
    2. Select the tables from your source endpoint
    3. Click OK.
  7. Click Save in the top-left portion of the window to save the task.

3. Run the Replication Task

Now you can run your newly defined replication task by clicking Run.

Once your task is running, you can:

  • Click Monitor to monitor the task.
  • Click Stop to stop the task.

4. Import the Data into Splice Machine

Once the files have been transferred use one of the ingest methods available in Splice Machine to import the data into your database.