features/monitor/add_targets.featureRunning behat features/monitor/add_targets.feature results in:
@rugged @workers @monitor-worker
Feature: A worker to dispatch `add-targets` tasks based on monitoring a directory for target files.
In order to allow a packaging pipeline not to hold credentials for the task queue,
As a TUF administrator
I need to ensure that a monitor worker can check a shared directory and dispatch `add-targets` tasks.
Background:
Given I reset Rugged
And I run the Rugged command "rugged --debug pause-processing --refreshing-timeout=0 --processing-timeout=0"
And I initialize a Rugged repo
And I run the Rugged command "rugged logs --truncate"
@scheduler @add-targets
Scenario: Dispatch `add-targets` task to targets-worker when new target is ready to add.
When I run the Rugged command "mkdir -p /opt/post_to_tuf/tuf_tmp_TIMESTAMP/"
And I run the Rugged command "fallocate -l 1K /opt/post_to_tuf/tuf_tmp_TIMESTAMP/foo.zip"
And I run the Rugged command "rugged resume-processing"
And I run the Rugged command "rugged logs --worker=monitor-worker --limit=0"
Then I should not get:
"""
New targets found in post-to-tuf directory:
"""
When I run the Rugged command "rugged logs --worker=monitor-worker --truncate"
And I run the Rugged command "mv /opt/post_to_tuf/tuf_tmp_TIMESTAMP/ /opt/post_to_tuf/tuf_ready_TIMESTAMP/"
And I wait (up to "30" seconds) for the file "/var/rugged/tuf_repo/metadata/targets.json" to contain "foo.zip"
And I run the Rugged command "rugged logs --worker=monitor-worker --limit=0"
Then I should get:
"""
New targets found in post-to-tuf directory:
tuf_ready_TIMESTAMP
Renaming 'tuf_ready_TIMESTAMP' to 'tuf_processing_TIMESTAMP'.
Dispatching 'add-targets' task for: tuf_processing_TIMESTAMP
"""
'Renaming 'tuf_ready_TIMESTAMP' to 'tuf_processing_TIMESTAMP'.' was not found in command output:
------
=== Log for monitor-worker: /var/log/rugged/rugged.log ===
2026-06-08 18:44:14,383 INFO (monitor-worker.find_new_targets_task): New content found in post-to-tuf directory.
2026-06-08 18:44:14,384 INFO (monitor-worker.find_new_targets_task): Count of post-to-tuf content: 1
2026-06-08 18:44:14,384 INFO (monitor-worker.find_new_targets_task): Age of oldest post-to-tuf content: 5 seconds
2026-06-08 18:44:14,384 INFO (monitor-worker.find_new_targets_task): New targets found in post-to-tuf directory:
2026-06-08 18:44:14,384 INFO (monitor-worker.find_new_targets_task): tuf_ready_TIMESTAMP
2026-06-08 18:44:14,384 INFO (monitor-worker.find_new_targets_task): Processing: tuf_ready_TIMESTAMP
2026-06-08 18:44:14,385 INFO (monitor-worker.find_new_targets_task): Dispatching 'add-targets' task for: tuf_processing_TIMESTAMP
2026-06-08 18:44:14,407 INFO (monitor-worker._dispatch_add_targets): Added the following targets to the repository:
foo.zip
Updated targets metadata.
------
(Exception)
And the "/var/rugged/incoming_targets/tuf_processing_TIMESTAMP" directory should not exist
When I run the Rugged command "rugged logs --worker=targets-worker --limit=0"
Then I should get:
"""
Received add-targets task.
Scanning for inbound targets in '/var/rugged/incoming_targets'
Found target: foo.zip
Moved '/var/rugged/incoming_targets/foo.zip' to '/var/rugged/tuf_repo/targets/foo.zip'
Moved inbound target 'foo.zip' to targets directory.
Added target 'foo.zip' to 'targets' role.
Signed 'targets' metadata with 'targets' key.
Wrote 'targets' metadata to file '/var/rugged/tuf_repo/metadata/targets.json'.
"""
When I run the Rugged command "rugged logs --worker=snapshot-worker --limit=0"
Then I should get:
"""
Received update-snapshot task.
Signed 'snapshot' metadata with 'snapshot' key.
Wrote 'snapshot' metadata to file '/var/rugged/tuf_repo/metadata/snapshot.json'.
Updated snapshot metadata.
"""
When I run the Rugged command "rugged logs --worker=timestamp-worker --limit=0"
Then I should get:
"""
Received update-timestamp task.
Signed 'timestamp' metadata with 'timestamp' key.
Wrote 'timestamp' metadata to file '/var/rugged/tuf_repo/metadata/timestamp.json'.
Updated timestamp metadata.
"""
When I run "cat /var/rugged/tuf_repo/metadata/targets.json"
Then I should get:
"""
"foo.zip": {
"""
And I should not get:
"""
tuf_
"""
When I run the Rugged command "rugged validate"
Then I should get:
"""
Metadata for the 'timestamp' role is valid.
Metadata for the 'snapshot' role is valid.
Metadata for the 'targets' role is valid.
"""
@scheduler @tuf-processing-dir @find-new-targets
Scenario: Ensure that multiple targets are supported.
Given I run "for i in $(seq -f '%03g' 0 4); do sudo sudo -u rugged mkdir -p /opt/post_to_tuf/tuf_tmp_TIMESTAMP$i/ ; done;"
And I run "for i in $(seq -f '%03g' 0 4); do sudo fallocate -l 1 /opt/post_to_tuf/tuf_tmp_TIMESTAMP$i/foo$i.zip ; done;"
And I run the Rugged command "rugged resume-processing"
When I run "for i in $(seq -f '%03g' 0 4); do sudo sudo -u rugged mv /opt/post_to_tuf/tuf_tmp_TIMESTAMP$i/ /opt/post_to_tuf/tuf_ready_TIMESTAMP$i/ ; done;"
And I wait (up to "60" seconds) for the file "/var/rugged/tuf_repo/metadata/targets.json" to contain "foo004.zip"
And I run the Rugged command "rugged logs --worker=monitor-worker --limit=0"
Then I should get:
"""
New targets found in post-to-tuf directory:
tuf_ready_TIMESTAMP000
tuf_ready_TIMESTAMP004
Renaming 'tuf_ready_TIMESTAMP000' to 'tuf_processing_TIMESTAMP000'.
Dispatching 'add-targets' task for: tuf_processing_TIMESTAMP000
"""
'Renaming 'tuf_ready_TIMESTAMP000' to 'tuf_processing_TIMESTAMP000'.' was not found in command output:
------
=== Log for monitor-worker: /var/log/rugged/rugged.log ===
2026-06-08 18:44:24,382 INFO (monitor-worker.find_new_targets_task): New content found in post-to-tuf directory.
2026-06-08 18:44:24,382 INFO (monitor-worker.find_new_targets_task): Count of post-to-tuf content: 5
2026-06-08 18:44:24,382 INFO (monitor-worker.find_new_targets_task): Age of oldest post-to-tuf content: 4 seconds
2026-06-08 18:44:24,382 INFO (monitor-worker.find_new_targets_task): New targets found in post-to-tuf directory:
2026-06-08 18:44:24,382 INFO (monitor-worker.find_new_targets_task): tuf_ready_TIMESTAMP000
2026-06-08 18:44:24,382 INFO (monitor-worker.find_new_targets_task): tuf_ready_TIMESTAMP001
2026-06-08 18:44:24,382 INFO (monitor-worker.find_new_targets_task): tuf_ready_TIMESTAMP002
2026-06-08 18:44:24,382 INFO (monitor-worker.find_new_targets_task): tuf_ready_TIMESTAMP003
2026-06-08 18:44:24,382 INFO (monitor-worker.find_new_targets_task): tuf_ready_TIMESTAMP004
2026-06-08 18:44:24,382 INFO (monitor-worker.find_new_targets_task): Processing: tuf_ready_TIMESTAMP000
2026-06-08 18:44:24,383 INFO (monitor-worker.find_new_targets_task): Dispatching 'add-targets' task for: tuf_processing_TIMESTAMP000
2026-06-08 18:44:24,400 INFO (monitor-worker._dispatch_add_targets): Added the following targets to the repository:
foo000.zip
Updated targets metadata.
2026-06-08 18:44:29,382 INFO (monitor-worker.find_new_targets_task): New content found in post-to-tuf directory.
2026-06-08 18:44:29,382 INFO (monitor-worker.find_new_targets_task): Count of post-to-tuf content: 4
2026-06-08 18:44:29,382 INFO (monitor-worker.find_new_targets_task): Age of oldest post-to-tuf content: 9 seconds
2026-06-08 18:44:29,383 INFO (monitor-worker.find_new_targets_task): New targets found in post-to-tuf directory:
2026-06-08 18:44:29,383 INFO (monitor-worker.find_new_targets_task): tuf_ready_TIMESTAMP001
2026-06-08 18:44:29,383 INFO (monitor-worker.find_new_targets_task): tuf_ready_TIMESTAMP002
2026-06-08 18:44:29,383 INFO (monitor-worker.find_new_targets_task): tuf_ready_TIMESTAMP003
2026-06-08 18:44:29,383 INFO (monitor-worker.find_new_targets_task): tuf_ready_TIMESTAMP004
2026-06-08 18:44:29,383 INFO (monitor-worker.find_new_targets_task): Processing: tuf_ready_TIMESTAMP001
2026-06-08 18:44:29,384 INFO (monitor-worker.find_new_targets_task): Dispatching 'add-targets' task for: tuf_processing_TIMESTAMP001
2026-06-08 18:44:29,402 INFO (monitor-worker._dispatch_add_targets): Added the following targets to the repository:
foo001.zip
Updated targets metadata.
2026-06-08 18:44:34,383 INFO (monitor-worker.find_new_targets_task): New content found in post-to-tuf directory.
2026-06-08 18:44:34,383 INFO (monitor-worker.find_new_targets_task): Count of post-to-tuf content: 4
2026-06-08 18:44:34,383 INFO (monitor-worker.find_new_targets_task): Age of oldest post-to-tuf content: 14 seconds
2026-06-08 18:44:34,383 INFO (monitor-worker.find_new_targets_task): Monitor worker is already processing targets. Waiting for next scan to continue.
2026-06-08 18:44:39,382 INFO (monitor-worker.find_new_targets_task): New content found in post-to-tuf directory.
2026-06-08 18:44:39,383 INFO (monitor-worker.find_new_targets_task): Count of post-to-tuf content: 4
2026-06-08 18:44:39,383 INFO (monitor-worker.find_new_targets_task): Age of oldest post-to-tuf content: 19 seconds
2026-06-08 18:44:39,383 INFO (monitor-worker.find_new_targets_task): Monitor worker is already processing targets. Waiting for next scan to continue.
2026-06-08 18:44:39,437 ERROR (logger.log_exception): TimeoutError thrown in send_task: The operation timed out.
2026-06-08 18:44:39,437 WARNING (task_queue.run_task): The operation timed out. Check status of timestamp-worker.
2026-06-08 18:44:39,437 ERROR (monitor-worker._dispatch_remote_task): Failed to update timestamp metadata on timestamp-worker.
2026-06-08 18:44:39,437 WARNING (monitor-worker._dispatch_remote_task): Task timed-out during update of timestamp metadata on timestamp-worker.
2026-06-08 18:44:39,437 INFO (monitor-worker._dispatch_remote_task): Retry 1/2: Retrying update of timestamp metadata on timestamp-worker.
2026-06-08 18:44:44,383 INFO (monitor-worker.find_new_targets_task): New content found in post-to-tuf directory.
2026-06-08 18:44:44,383 INFO (monitor-worker.find_new_targets_task): Count of post-to-tuf content: 3
2026-06-08 18:44:44,384 INFO (monitor-worker.find_new_targets_task): Age of oldest post-to-tuf content: 24 seconds
2026-06-08 18:44:44,384 INFO (monitor-worker.find_new_targets_task): New targets found in post-to-tuf directory:
2026-06-08 18:44:44,384 INFO (monitor-worker.find_new_targets_task): tuf_ready_TIMESTAMP002
2026-06-08 18:44:44,384 INFO (monitor-worker.find_new_targets_task): tuf_ready_TIMESTAMP003
2026-06-08 18:44:44,384 INFO (monitor-worker.find_new_targets_task): tuf_ready_TIMESTAMP004
2026-06-08 18:44:44,384 INFO (monitor-worker.find_new_targets_task): Processing: tuf_ready_TIMESTAMP002
2026-06-08 18:44:44,385 INFO (monitor-worker.find_new_targets_task): Dispatching 'add-targets' task for: tuf_processing_TIMESTAMP002
2026-06-08 18:44:44,403 INFO (monitor-worker._dispatch_add_targets): Added the following targets to the repository:
foo002.zip
Updated targets metadata.
2026-06-08 18:44:49,383 INFO (monitor-worker.find_new_targets_task): New content found in post-to-tuf directory.
2026-06-08 18:44:49,383 INFO (monitor-worker.find_new_targets_task): Count of post-to-tuf content: 2
2026-06-08 18:44:49,383 INFO (monitor-worker.find_new_targets_task): Age of oldest post-to-tuf content: 29 seconds
2026-06-08 18:44:49,384 INFO (monitor-worker.find_new_targets_task): New targets found in post-to-tuf directory:
2026-06-08 18:44:49,384 INFO (monitor-worker.find_new_targets_task): tuf_ready_TIMESTAMP003
2026-06-08 18:44:49,384 INFO (monitor-worker.find_new_targets_task): tuf_ready_TIMESTAMP004
2026-06-08 18:44:49,384 INFO (monitor-worker.find_new_targets_task): Processing: tuf_ready_TIMESTAMP003
2026-06-08 18:44:49,385 INFO (monitor-worker.find_new_targets_task): Dispatching 'add-targets' task for: tuf_processing_TIMESTAMP003
2026-06-08 18:44:49,403 INFO (monitor-worker._dispatch_add_targets): Added the following targets to the repository:
foo003.zip
Updated targets metadata.
2026-06-08 18:44:54,382 INFO (monitor-worker.find_new_targets_task): New content found in post-to-tuf directory.
2026-06-08 18:44:54,382 INFO (monitor-worker.find_new_targets_task): Count of post-to-tuf content: 1
2026-06-08 18:44:54,382 INFO (monitor-worker.find_new_targets_task): Age of oldest post-to-tuf content: 34 seconds
2026-06-08 18:44:54,383 INFO (monitor-worker.find_new_targets_task): New targets found in post-to-tuf directory:
2026-06-08 18:44:54,383 INFO (monitor-worker.find_new_targets_task): tuf_ready_TIMESTAMP004
2026-06-08 18:44:54,383 INFO (monitor-worker.find_new_targets_task): Processing: tuf_ready_TIMESTAMP004
2026-06-08 18:44:54,384 INFO (monitor-worker.find_new_targets_task): Dispatching 'add-targets' task for: tuf_processing_TIMESTAMP004
2026-06-08 18:44:54,401 INFO (monitor-worker._dispatch_add_targets): Added the following targets to the repository:
foo004.zip
Updated targets metadata.
------
(Exception)
And the "/var/rugged/incoming_targets/tuf_processing_TIMESTAMP000" directory should not exist
When I run "cat /var/rugged/tuf_repo/metadata/targets.json"
Then I should get:
"""
"foo000.zip": {
"foo001.zip": {
"foo002.zip": {
"foo003.zip": {
"foo004.zip": {
"""
And I should not get:
"""
tuf_
"""
When I run the Rugged command "rugged validate"
Then I should get:
"""
Metadata for the 'root' role is valid.
Metadata for the 'timestamp' role is valid.
Metadata for the 'snapshot' role is valid.
Metadata for the 'targets' role is valid.
"""
@scheduler @tuf-processing-dir @find-new-targets
Scenario: Ensure that targets in directories are supported.
Given I run "for i in $(seq -f '%03g' 0 1); do sudo sudo -u rugged mkdir -p /opt/post_to_tuf/tuf_tmp_TIMESTAMP$i/dir/ ; done;"
And I run "for i in $(seq -f '%03g' 0 1); do sudo fallocate -l 1 /opt/post_to_tuf/tuf_tmp_TIMESTAMP$i/dir/foo$i.zip ; done;"
And I run the Rugged command "rugged resume-processing"
And I run the Rugged command "rugged logs --worker=monitor-worker --truncate"
When I run "for i in $(seq -f '%03g' 0 1); do sudo sudo -u rugged mv /opt/post_to_tuf/tuf_tmp_TIMESTAMP$i/ /opt/post_to_tuf/tuf_ready_TIMESTAMP$i/ ; done;"
And I wait (up to "30" seconds) for the file "/var/rugged/tuf_repo/metadata/targets.json" to contain "foo001.zip"
And I run the Rugged command "rugged logs --worker=monitor-worker --limit=0"
Then I should get:
"""
New targets found in post-to-tuf directory:
tuf_ready_TIMESTAMP000
tuf_ready_TIMESTAMP001
Renaming 'tuf_ready_TIMESTAMP000' to 'tuf_processing_TIMESTAMP000'.
Dispatching 'add-targets' task for: tuf_processing_TIMESTAMP000
"""
'Renaming 'tuf_ready_TIMESTAMP000' to 'tuf_processing_TIMESTAMP000'.' was not found in command output:
------
=== Log for monitor-worker: /var/log/rugged/rugged.log ===
2026-06-08 18:45:04,383 INFO (monitor-worker.find_new_targets_task): New content found in post-to-tuf directory.
2026-06-08 18:45:04,383 INFO (monitor-worker.find_new_targets_task): Count of post-to-tuf content: 2
2026-06-08 18:45:04,383 INFO (monitor-worker.find_new_targets_task): Age of oldest post-to-tuf content: 3 seconds
2026-06-08 18:45:04,384 INFO (monitor-worker.find_new_targets_task): New targets found in post-to-tuf directory:
2026-06-08 18:45:04,384 INFO (monitor-worker.find_new_targets_task): tuf_ready_TIMESTAMP000
2026-06-08 18:45:04,384 INFO (monitor-worker.find_new_targets_task): tuf_ready_TIMESTAMP001
2026-06-08 18:45:04,384 INFO (monitor-worker.find_new_targets_task): Processing: tuf_ready_TIMESTAMP000
2026-06-08 18:45:04,385 INFO (monitor-worker.find_new_targets_task): Dispatching 'add-targets' task for: tuf_processing_TIMESTAMP000
2026-06-08 18:45:04,401 INFO (monitor-worker._dispatch_add_targets): Added the following targets to the repository:
dir/foo000.zip
Updated targets metadata.
2026-06-08 18:45:09,383 INFO (monitor-worker.find_new_targets_task): New content found in post-to-tuf directory.
2026-06-08 18:45:09,384 INFO (monitor-worker.find_new_targets_task): Count of post-to-tuf content: 1
2026-06-08 18:45:09,384 INFO (monitor-worker.find_new_targets_task): Age of oldest post-to-tuf content: 8 seconds
2026-06-08 18:45:09,384 INFO (monitor-worker.find_new_targets_task): New targets found in post-to-tuf directory:
2026-06-08 18:45:09,384 INFO (monitor-worker.find_new_targets_task): tuf_ready_TIMESTAMP001
2026-06-08 18:45:09,385 INFO (monitor-worker.find_new_targets_task): Processing: tuf_ready_TIMESTAMP001
2026-06-08 18:45:09,386 INFO (monitor-worker.find_new_targets_task): Dispatching 'add-targets' task for: tuf_processing_TIMESTAMP001
2026-06-08 18:45:09,403 INFO (monitor-worker._dispatch_add_targets): Added the following targets to the repository:
dir/foo001.zip
Updated targets metadata.
------
(Exception)
And the "/var/rugged/incoming_targets/tuf_processing_TIMESTAMP000" directory should not exist
When I run "cat /var/rugged/tuf_repo/metadata/targets.json"
Then I should get:
"""
"dir/foo000.zip": {
"dir/foo001.zip": {
"""
When I run the Rugged command "rugged validate"
Then I should get:
"""
Metadata for the 'root' role is valid.
Metadata for the 'timestamp' role is valid.
Metadata for the 'snapshot' role is valid.
Metadata for the 'targets' role is valid.
"""
@scheduler @tuf-processing-dir
Scenario: Ensure that errant `tuf_` directories (in `inbound_targets`) are ignored when scanning for targets.
When I run the Rugged command "mkdir -p /opt/post_to_tuf/tuf_tmp_TIMESTAMP/"
And I run the Rugged command "fallocate -l 1K /opt/post_to_tuf/tuf_tmp_TIMESTAMP/foo.zip"
When I run the Rugged command "mkdir -p /var/rugged/incoming_targets/tuf_ready_TIMESTAMP3/"
And I run the Rugged command "fallocate -l 1K /var/rugged/incoming_targets/tuf_ready_TIMESTAMP3/foo3.zip"
When I run the Rugged command "mkdir -p /var/rugged/incoming_targets/tuf_processing_TIMESTAMP4/"
And I run the Rugged command "fallocate -l 1K /var/rugged/incoming_targets/tuf_processing_TIMESTAMP4/foo4.zip"
When I run the Rugged command "rugged logs --worker=monitor-worker --truncate"
And I run the Rugged command "mv /opt/post_to_tuf/tuf_tmp_TIMESTAMP/ /opt/post_to_tuf/tuf_ready_TIMESTAMP/"
And I run the Rugged command "rugged resume-processing"
And I wait (up to "30" seconds) for the file "/var/rugged/tuf_repo/metadata/targets.json" to contain "foo.zip"
When I run "cat /var/rugged/tuf_repo/metadata/targets.json"
Then I should get:
"""
"foo.zip": {
"""
And I should not get:
"""
tuf_
"foo3.zip": {
"foo4.zip": {
"""
When I run the Rugged command "rugged validate"
Then I should get:
"""
Metadata for the 'root' role is valid.
Metadata for the 'timestamp' role is valid.
Metadata for the 'snapshot' role is valid.
Metadata for the 'targets' role is valid.
"""
@scheduler @tuf-processing-dir @find-new-targets @gitlab-149
Scenario: Ensure that empty directories are cleaned up.
Given I run the Rugged command "mkdir -p /opt/post_to_tuf/tuf_ready_TIMESTAMP/empty_dir/"
Given I run the Rugged command "mkdir -p /opt/post_to_tuf/tuf_ready_TIMESTAMP/deepy/nested/empty/dir/"
And I run the Rugged command "fallocate -l 1K /opt/post_to_tuf/tuf_ready_TIMESTAMP/foo.zip"
And I run the Rugged command "rugged resume-processing"
When I wait (up to "30" seconds) for the file "/var/rugged/tuf_repo/metadata/targets.json" to contain "foo.zip"
Then the "/var/rugged/incoming_targets/empty_dir" directory should not exist
And the "/var/rugged/incoming_targets/deeply" directory should not exist
And the "/var/rugged/incoming_targets" directory should exist
--- Failed scenarios:
features/monitor/add_targets.feature:33
features/monitor/add_targets.feature:101
features/monitor/add_targets.feature:140
5 scenarios (2 passed, 3 failed)
91 steps (65 passed, 3 failed, 23 skipped)