features/monitor/pause_resume.featureRunning behat features/monitor/pause_resume.feature results in:
@rugged @workers @monitor-worker
Feature: Scheduled tasks respect pause/resume commands.
In order to cleanly redeploy Rugged and TUF Root metadata
As a TUF administrator
I need to be able to pause and resume scheduled tasks.
Background:
Given I reset Rugged
And I run "sudo rm -rf /opt/post_to_tuf/*"
@scheduler @pause-processing
Scenario: The pause-processing semaphore blocks scheduled target scans.
Given I run the Rugged command "rugged pause-processing --refreshing-timeout=0 --processing-timeout=0"
And I initialize a Rugged repo
And I record the expiry timestamp from "targets.json"
When I run the Rugged command "mkdir -p /opt/post_to_tuf/tuf_tmp_TIMESTAMP/"
And I run the Rugged command "fallocate -l 1K /opt/post_to_tuf/tuf_tmp_TIMESTAMP/foo.zip"
And I run the Rugged command "mv /opt/post_to_tuf/tuf_tmp_TIMESTAMP/ /opt/post_to_tuf/tuf_ready_TIMESTAMP/"
And I wait "5" seconds
And I run the Rugged command "rugged logs --worker=monitor-worker --limit=0"
Then I should get:
"""
Detected pause-processing flag at: /opt/post_to_tuf/tuf_paused
The monitor worker is paused.
"""
'Detected pause-processing flag at: /opt/post_to_tuf/tuf_paused' was not found in command output:
------
=== Log for monitor-worker: /var/log/rugged/rugged.log ===
2026-06-08 18:42:54,381 WARNING (monitor-worker.find_new_targets_task): The monitor worker is paused.
2026-06-08 18:42:54,381 INFO (monitor-worker.find_new_targets_task): Skipping find-new-targets task.
------
(Exception)
And I should not get:
"""
New targets found in post-to-tuf directory:
tuf_ready_TIMESTAMP
"""
Then the "/opt/post_to_tuf/tuf_ready_TIMESTAMP" directory should exist
When I run the Rugged command "rugged logs --worker=targets-worker --limit=0"
Then I should not get:
"""
Received add-targets task.
"""
And the expiry timestamp from "targets.json" has not changed
@refresh-expiry @pause-processing
Scenario: The pause-processing semaphore blocks scheduled expiry refresh tasks.
Given I run the Rugged command "rugged pause-processing --refreshing-timeout=0 --processing-timeout=0"
And I run the Rugged command "cp features/fixtures/config/expiry/short_timestamp_expiry.yaml /var/rugged/.config/rugged/config.yaml"
And I initialize a Rugged repo
And I record the expiry timestamp from "timestamp.json"
And I wait "5" seconds
When I run the Rugged command "rugged logs --worker=monitor-worker --limit=0"
Then I should get:
"""
Detected pause-processing flag at: /opt/post_to_tuf/tuf_paused
The monitor worker is paused.
"""
'Detected pause-processing flag at: /opt/post_to_tuf/tuf_paused' was not found in command output:
------
=== Log for monitor-worker: /var/log/rugged/rugged.log ===
2026-06-08 18:43:04,381 WARNING (monitor-worker.find_new_targets_task): The monitor worker is paused.
2026-06-08 18:43:04,381 INFO (monitor-worker.find_new_targets_task): Skipping find-new-targets task.
------
(Exception)
And I should not get:
"""
Fetching expiring metadata from workers.
"""
When I run the Rugged command "rugged logs --worker=timestamp-worker --limit=0"
Then I should not get:
"""
'timestamp.json' metadata expiry is imminent
"""
And the expiry timestamp from "timestamp.json" has not changed
@scheduler @resume-processing
Scenario: The resume-processing semaphore allows scheduled target scans to start.
Given I run the Rugged command "rugged pause-processing --refreshing-timeout=0 --processing-timeout=0"
And I initialize a Rugged repo
And I record the expiry timestamp from "targets.json"
And I run the Rugged command "rm -rf /opt/post_to_tuf/tuf_refreshing_expiry"
And I run the Rugged command "rugged pause-processing"
When I run the Rugged command "mkdir -p /opt/post_to_tuf/tuf_tmp_TIMESTAMP/"
And I run the Rugged command "fallocate -l 1K /opt/post_to_tuf/tuf_tmp_TIMESTAMP/foo.zip"
And I run the Rugged command "mv /opt/post_to_tuf/tuf_tmp_TIMESTAMP/ /opt/post_to_tuf/tuf_ready_TIMESTAMP/"
And I run the Rugged command "rugged resume-processing"
And I wait "5" seconds
And I run the Rugged command "rugged logs --worker=monitor-worker --limit=0"
Then I should get:
"""
New targets found in post-to-tuf directory:
tuf_ready_TIMESTAMP
"""
Then the "/opt/post_to_tuf/tuf_ready_TIMESTAMP" directory should not exist
When I run the Rugged command "rugged logs --worker=targets-worker --limit=0"
Then I should get:
"""
Received add-targets task.
"""
And the expiry timestamp from "targets.json" has changed
@refresh-expiry @resume-processing
Scenario: The resume-processing semaphore allows scheduled expiry refresh tasks to start.
Given I run the Rugged command "rugged pause-processing --refreshing-timeout=0 --processing-timeout=0"
And I run the Rugged command "cp features/fixtures/config/expiry/short_timestamp_expiry.yaml /var/rugged/.config/rugged/config.yaml"
And I initialize a Rugged repo
And I record a reference hash of "/var/rugged/tuf_repo/metadata/timestamp.json"
And I record the expiry timestamp from "timestamp.json"
And I run the Rugged command "rugged resume-processing"
When I wait (up to "15" seconds) for the file "/var/rugged/tuf_repo/metadata/timestamp.json" to change
When I run the Rugged command "rugged logs --worker=monitor-worker --limit=0"
Then I should get:
"""
Fetching expiring metadata from workers.
"""
'Fetching expiring metadata from workers.' was not found in command output:
------
=== Log for monitor-worker: /var/log/rugged/rugged.log ===
2026-06-08 18:43:24,383 WARNING (monitor-worker.find_new_targets_task): The monitor worker is paused.
2026-06-08 18:43:24,384 INFO (monitor-worker.find_new_targets_task): Skipping find-new-targets task.
2026-06-08 18:43:29,431 ERROR (logger.log_exception): TimeoutError thrown in send_task: The operation timed out.
2026-06-08 18:43:29,431 WARNING (task_queue.run_task): The operation timed out. Check status of snapshot-worker.
2026-06-08 18:43:29,431 ERROR (monitor-worker._dispatch_remote_task): Failed to update snapshot metadata on snapshot-worker.
2026-06-08 18:43:29,431 WARNING (monitor-worker._dispatch_remote_task): Task timed-out during update of snapshot metadata on snapshot-worker.
2026-06-08 18:43:29,431 INFO (monitor-worker._dispatch_remote_task): Retry 1/2: Retrying update of snapshot metadata on snapshot-worker.
2026-06-08 18:43:29,464 ERROR (logger.log_exception): FileNotFoundError thrown in find_new_targets_task: [Errno 2] No such file or directory: '/opt/post_to_tuf/tuf_processing_TIMESTAMP'
2026-06-08 18:43:29,464 ERROR (monitor-worker.find_new_targets_task): Error removing directory '/opt/post_to_tuf/tuf_processing_TIMESTAMP'
------
(Exception)
When I run the Rugged command "rugged logs --worker=timestamp-worker --limit=0"
Then I should get:
"""
'timestamp.json' metadata expiry is imminent
"""
And the expiry timestamp from "timestamp.json" has changed
--- Failed scenarios:
features/monitor/pause_resume.feature:31
features/monitor/pause_resume.feature:59
features/monitor/pause_resume.feature:111
4 scenarios (1 passed, 3 failed)
61 steps (46 passed, 3 failed, 12 skipped)