无法在Rails测试环境中使用TimescaleDB

2023-12-29

我被困在 Rails 中使用 TimescaleDB - 开发中一切正常，但在我的测试套件中我无法插入任何数据。

我尝试过的

A) 使用 SQL 模式转储

这导致了我看到的原始错误消息。它确实为 TimescaleDB 创建了部分架构，但不是全部。我有一个超表，但它无法正常工作

B) 使用 Ruby 架构转储

这让我可以插入到我的表中，但它根本不是超表 - ruby 语法丢失了与 TimescaleDB 和超表相关的所有内容。

C) 直接迁移测试数据库

我尝试避免 schema.struct 转储并加载以下内容：

$ rails db:drop
Dropped database 'my_app_development'
Dropped database 'my_app_test'
$ RAILS_ENV=test rails db:create
Created database 'my_app_test'
$ RAILS_ENV=test rails db:migrate
== 20200517164444 EnableTimescaledbExtension: migrating =======================
-- enable_extension("timescaledb")
WARNING:  
WELCOME TO
 _____ _                               _     ____________  
|_   _(_)                             | |    |  _  \ ___ \ 
  | |  _ _ __ ___   ___  ___  ___ __ _| | ___| | | | |_/ / 
  | | | |  _ ` _ \ / _ \/ __|/ __/ _` | |/ _ \ | | | ___ \ 
  | | | | | | | | |  __/\__ \ (_| (_| | |  __/ |/ /| |_/ /
  |_| |_|_| |_| |_|\___||___/\___\__,_|_|\___|___/ \____/
               Running version 1.7.0
For more information on TimescaleDB, please visit the following links:

 1. Getting started: https://docs.timescale.com/getting-started
 2. API reference documentation: https://docs.timescale.com/api
 3. How TimescaleDB is designed: https://docs.timescale.com/introduction/architecture

Note: TimescaleDB collects anonymous reports to better understand and assist our users.
For more information and how to disable, please see our docs https://docs.timescaledb.com/using-timescaledb/telemetry.

   -> 0.2315s
== 20200517164444 EnableTimescaledbExtension: migrated (0.2316s) ==============

== 20200517165027 CreateAccounts: migrating ===================================
-- create_table(:accounts)
   -> 0.0095s
== 20200517165027 CreateAccounts: migrated (0.0095s) ==========================

== 20200517165103 CreateMetrics: migrating ====================================
-- create_table(:metrics)
   -> 0.0116s
== 20200517165103 CreateMetrics: migrated (0.0117s) ===========================

== 20200517170842 CreateEvents: migrating =====================================
-- create_table(:events)
   -> 0.0072s
-- remove_column(:events, :id)
   -> 0.0020s
-- execute("SELECT create_hypertable('events', 'time');\n")
   -> 0.0047s
== 20200517170842 CreateEvents: migrated (0.0142s) ============================

pg_dump: warning: there are circular foreign-key constraints on this table:
pg_dump:   hypertable
pg_dump: You might not be able to restore the dump without using --disable-triggers or temporarily dropping the constraints.
pg_dump: Consider using a full dump instead of a --data-only dump to avoid this problem.
pg_dump: warning: there are circular foreign-key constraints on this table:
pg_dump:   chunk
pg_dump: You might not be able to restore the dump without using --disable-triggers or temporarily dropping the constraints.
pg_dump: Consider using a full dump instead of a --data-only dump to avoid this problem.

但是当运行测试套件时，它与尝试 A 相同。

在实际打印此消息几次后运行测试，这让我认为 Rails 会自动神奇地使用structure.sql再次重新创建测试数据库：


psql:/home/axel/src/my_app/db/structure.sql:16: WARNING:  
WELCOME TO
 _____ _                               _     ____________  
|_   _(_)                             | |    |  _  \ ___ \ 
  | |  _ _ __ ___   ___  ___  ___ __ _| | ___| | | | |_/ / 
  | | | |  _ ` _ \ / _ \/ __|/ __/ _` | |/ _ \ | | | ___ \ 
  | | | | | | | | |  __/\__ \ (_| (_| | |  __/ |/ /| |_/ /
  |_| |_|_| |_| |_|\___||___/\___\__,_|_|\___|___/ \____/
               Running version 1.7.0
For more information on TimescaleDB, please visit the following links:

 1. Getting started: https://docs.timescale.com/getting-started
 2. API reference documentation: https://docs.timescale.com/api
 3. How TimescaleDB is designed: https://docs.timescale.com/introduction/architecture

Note: TimescaleDB collects anonymous reports to better understand and assist our users.
For more information and how to disable, please see our docs https://docs.timescaledb.com/using-timescaledb/telemetry.

错误信息

$ rails test
Running via Spring preloader in process 107937
Run options: --seed 29840

# Running:

E

Error:
Api::EventsControllerTest#test_POST_event_data_-_new_metric:
DRb::DRbRemoteError: PG::FeatureNotSupported: ERROR:  invalid INSERT on the root table of hypertable "events"
HINT:  Make sure the TimescaleDB extension has been preloaded.
 (ActiveRecord::StatementInvalid)
    app/controllers/api/events_controller.rb:5:in `create'
    test/controllers/api/events_controller_test.rb:9:in `block in <class:EventsControllerTest>'


rails test test/controllers/api/events_controller_test.rb:8



Finished in 0.215286s, 4.6450 runs/s, 0.0000 assertions/s.
1 runs, 0 assertions, 0 failures, 1 errors, 0 skips

我感觉这与 Rails 如何使用创建测试数据库有关schema.rb（对于默认的config.active_record.schema_format = :ruby) or structure.sql (for config.active_record.schema_format = :sql。我已经尝试了结构的 Ruby 和 SQL 设置，但都不起作用 - 开发数据库已正确迁移，但测试数据库未正确设置。

在下面的两个数据库（开发和测试）中，我们可以看到唯一的区别是测试数据库丢失了：Child tables: _timescaledb_internal._hyper_1_1_chunk

开发数据库

$ psql -d my_app_development
psql (12.2)
Type "help" for help.

my_app_development=# SHOW shared_preload_libraries;
 shared_preload_libraries 
--------------------------
 timescaledb
(1 row)

my_app_development=# insert into events (metric_id, time, value) VALUES (1, NOW(), 22);
INSERT 0 1
my_app_development=# \d+ events
                                              Table "public.events"
  Column   |            Type             | Collation | Nullable | Default | Storage | Stats target | Description 
-----------+-----------------------------+-----------+----------+---------+---------+--------------+-------------
 metric_id | bigint                      |           |          |         | plain   |              | 
 time      | timestamp without time zone |           | not null |         | plain   |              | 
 value     | numeric                     |           |          |         | main    |              | 
Indexes:
    "events_time_idx" btree ("time" DESC)
Triggers:
    ts_insert_blocker BEFORE INSERT ON events FOR EACH ROW EXECUTE FUNCTION _timescaledb_internal.insert_blocker()
Child tables: _timescaledb_internal._hyper_1_1_chunk
Access method: heap

Test DB

$ psql -d my_app_test
psql (12.2)
Type "help" for help.

my_app_test=# SHOW shared_preload_libraries;
 shared_preload_libraries 
--------------------------
 timescaledb
(1 row)

my_app_test=# insert into events (metric_id, time, value) VALUES (1, NOW(), 22);
ERROR:  invalid INSERT on the root table of hypertable "events"
HINT:  Make sure the TimescaleDB extension has been preloaded.
my_app_test=# \d+ events
                                              Table "public.events"
  Column   |            Type             | Collation | Nullable | Default | Storage | Stats target | Description 
-----------+-----------------------------+-----------+----------+---------+---------+--------------+-------------
 metric_id | bigint                      |           |          |         | plain   |              | 
 time      | timestamp without time zone |           | not null |         | plain   |              | 
 value     | numeric                     |           |          |         | main    |              | 
Indexes:
    "events_time_idx" btree ("time" DESC)
Triggers:
    ts_insert_blocker BEFORE INSERT ON events FOR EACH ROW EXECUTE FUNCTION _timescaledb_internal.insert_blocker()
Access method: heap

具有 SQL 架构的 ActiveRecord

CREATE EXTENSION IF NOT EXISTS timescaledb WITH SCHEMA public;

SET default_tablespace = '';

SET default_table_access_method = heap;

CREATE TABLE public.events (
    metric_id bigint,
    "time" timestamp without time zone NOT NULL,
    value numeric
);

CREATE INDEX events_time_idx ON public.events USING btree ("time" DESC);

CREATE TRIGGER ts_insert_blocker BEFORE INSERT ON public.events FOR EACH ROW EXECUTE FUNCTION _timescaledb_internal.insert_blocker();

带有 Ruby 模式的 ActiveRecord

ActiveRecord::Schema.define(version: 2020_05_17_170842) do

  # These are extensions that must be enabled in order to support this database
  enable_extension "plpgsql"
  enable_extension "timescaledb"

  create_table "events", id: false, force: :cascade do |t|
    t.bigint "metric_id"
    t.datetime "time", null: false
    t.decimal "value"
    t.index ["time"], name: "events_time_idx", order: :desc
  end
end

注意：这会丢失ts_insert_blocker触发器并让我插入events表，但它不再是超表：


my_app_test=# \d+ events
                                              Table "public.events"
  Column   |            Type             | Collation | Nullable | Default | Storage | Stats target | Description 
-----------+-----------------------------+-----------+----------+---------+---------+--------------+-------------
 metric_id | bigint                      |           |          |         | plain   |              | 
 time      | timestamp without time zone |           | not null |         | plain   |              | 
 value     | numeric                     |           |          |         | main    |              | 
Indexes:
    "events_time_idx" btree ("time" DESC)
Access method: heap

附加信息

相关问题：使用 Rails 4.2 针对 TimescaleDB 数据库运行 RSpec 测试套件 https://stackoverflow.com/questions/59141045/running-an-rspec-test-suite-against-a-timescaledb-database-with-rails-4-2- 这些建议对我不起作用，并且没有可接受的答案。

版本信息：

导轨 6.0.3
Postgres 12.2
时间刻度数据库 1.7.0

Edit 1

我将以下内容添加到我的test/test_helper.rb类似于@cstabru提到的解决方法

def execute_create_hypertable(sql)
  ActiveRecord::Base.connection.execute(sql)
rescue ActiveRecord::StatementInvalid => e
  raise e unless e.message.include? 'is already a hypertable'
end

execute_create_hypertable <<~SQL
  SELECT create_hypertable('events', 'time');
SQL

但也许我们可以使用类似的东西SELECT create_hypertable('hypertable_name', 'time_field', if_not_exists => TRUE在初始化程序中而不是在数据库迁移中创建超表？

我也遇到了这个问题，无论我以哪种方式重新创建数据库模式（sql 或 ruby 格式），都不会重新创建超级表，因为未导出时间刻度内部模式数据。

注意到当我使用sql格式，它复制ts_insert_blocker触发器确实会因此错误而中断表上的插入（我相信是由于触发器功能不可用）

PG::FeatureNotSupported: ERROR:  invalid INSERT on the root table of hypertable "hypertable_name"
HINT:  Make sure the TimescaleDB extension has been preloaded.

要解决根本问题（sql 或 ruby 格式），我们可以通过以下方式手动重新创建超表（并删除触发器）

DROP TRIGGER IF EXISTS ts_insert_blocker ON events;
DROP TRIGGER

SELECT create_hypertable('hypertable_name', 'time_field', if_not_exists => TRUE);
....
(1 row)

现在手动检查超表是否存在，因为https://github.com/timescale/timescaledb/pull/862 https://github.com/timescale/timescaledb/pull/862

SELECT * FROM timescaledb_information.hypertable;

我已将这些 DDL 命令添加到我的spec_helper.rb确保测试数据库使用实际的超表。我想确保测试数据库架构反映我的生产/暂存设置。

config.before(:suite) do
  # ensure the hypertable_name hypertable is setup correctly
  ActiveRecord::Base.connection.execute(
    "DROP TRIGGER IF EXISTS ts_insert_blocker ON hypertable_name;"
  )
  ActiveRecord::Base.connection.execute(
    "SELECT create_hypertable('hypertable_name', 'time_field', if_not_exists => TRUE);"
  )
  has_hypertables_sql = "SELECT * FROM timescaledb_information.hypertable WHERE table_name = 'hypertable_name';"
  if ActiveRecord::Base.connection.execute(has_hypertables_sql).to_a.empty?
    raise "TimescaleDB missing hypertable on 'hypertable_name' table"
  end
end

如果人们觉得这很有用，我可以考虑提取到 gem 来帮助 Rails 环境的模式恢复，https://github.com/timescale/timescaledb/issues/1916 https://github.com/timescale/timescaledb/issues/1916

本文内容由网友自发贡献，版权归原作者所有，本站不承担相应法律责任。如您发现有涉嫌抄袭侵权的内容，请联系:hwhale#tublm.com(使用前将#替换为@)