从 MySQL 到 TSDB 数据迁移

by BenedictJin 4 years ago

macOS ◆ xterm-256color ◆ zsh 5528 views

# 首先，检查 MySQL 和 TSDB 是否正常运行
mysql --version
curl http://localhost:8242/api/version

# 确认版本是否支持
# MySQL 只保证 5.x 是兼容的
# TSDB 只保证 2.4.x 及以上版本是兼容的

# 下载
cd /tmp/datax
wget https://tsdbtools.oss-cn-hangzhou.aliyuncs.com/datax_mysql_2_tsdb.tar.gz
tar zxvf datax_mysql_2_tsdb.tar.gz

# 配置任务
cd /tmp/datax
cp datax/job/job.json mysql2tsdb.json
{
  "job": {
    "content": [
      {
        "reader": {
          "name": "mysqlreader",
          "parameter": {
            "column": [
              "name",
              "type",
              "create_time",
              "price"
            ],
            "connection": [
              {
                "jdbcUrl": [
                  "jdbc:mysql://127.0.0.1:3306/datax"
                ],
                "table": [
                  "book"
                ]
              }
            ],
            "password": "yuzhouwan",
            "splitPk": "id",
            "username": "root"
          }
        },
        "writer": {
          "name": "tsdbwriter",
          "parameter": {
            "column": [
              "name",
              "type",
              "create_time",
              "price"
            ],
            "columnType": [
              "tag",
              "tag",
              "timestamp",
              "metric_num"
            ],
            "sourceDbType": "RDB",
            "tsdbAddress": "localhost",
            "tsdbPort": 8242
          }
        }
      }
    ],
    "setting": {
      "speed": {
        "channel": 1
      }
    }
  }
}

# 启动之前，确认 MySQL 存在数据，而 TSDB 不存在数据
# MySQL
echo "use datax; select * from book;" | mysql -u root -p

# TSDB
curl -X POST -H "Content-Type: application/json " localhost:8240/api/query -d '{"start":1556527068,"end":1556527070,"queries":[{"aggregator":"none","metric":"price"}]}'


# 通过 datax.py 脚本启动任务
python datax/bin/datax.py mysql2tsdb.json
# 任务已完成
# 现在，我们来检查是否写入成功
# MySQL
echo "use datax; select * from book;" | mysql -u root -p
# TSDB
curl -X POST -H "Content-Type: application/json " localhost:8242/api/query -d '{"start":1556527068,"end":1556527070,"queries":[{"aggregator":"none","metric":"price"}]}'

# 恭喜！
# 数据已成功迁移！

More by BenedictJin

从 Prometheus 到 TSDB 数据迁移 08:08

by BenedictJin 4 years ago

数据迁移 Quick Start 04:48

by BenedictJin 4 years ago

宇宙湾 00:20

by BenedictJin 4 years ago

从 OpenTSDB 到 TSDB 数据迁移 08:18

by BenedictJin 4 years ago

See all

You can download this recording in asciicast v2 format, as a .cast file.

Download

Replay in terminal

You can replay the downloaded recording in your terminal using the asciinema play command:

asciinema play 247493.cast

If you don't have asciinema CLI installed then see installation instructions.

Use with stand-alone player on your website

Download asciinema player from the releases page (you only need .js and .css file), then use it like this:

<!DOCTYPE html>
<html>
<head>
  <link rel="stylesheet" type="text/css" href="asciinema-player.css" />
</head>
<body>
  <div id="player"></div>
  <script src="asciinema-player.min.js"></script>
  <script>
    AsciinemaPlayer.create(
      '/assets/247493.cast',
      document.getElementById('player'),
      { cols: 178, rows: 48 }
    );
  </script>
</body>
</html>

See asciinema player quick-start guide for full usage instructions.

While this site doesn't offer GIF conversion at the moment, you can still do it yourself with the help of asciinema GIF generator utility - agg.

Once you have it installed run the following command to create GIF file:

agg https://asciinema.org/a/247493 247493.gif

Or, if you already downloaded the recording file:

agg 247493.cast 247493.gif

Check agg --help for all available options. You can change font family and size, select color theme, adjust speed and more.

See agg manual for full usage instructions.