KBEC-00235 - Removing duplicate jobIds to unjam the background job deleter

Description

You notice deleted jobs remain.

You notice Sentry jobs accumulating on the jobs page. Normally there is only one Sentry job as Sentry cleans up after itself.

and

You find messages of the following form in the commander.log

2011-07-07T16:56:34.597 | ERROR | Workflow-003         |   69528 | 1396873 | backgroundDelete                              | OperationInvoker               | Operation exception: org.hibernate.NonUniqueResultException: query did not return a unique result: 2org.hibernate.NonUniqueResultException: query did not return a unique result: 2

Solution

This solution involves running queries in your Database. It is recommended that only DBA’s or users with experience running queries use these solutions.

The following solution can be done when CloudBees CD (CloudBees Flow) is running.

Find the results of this SQL query:

 SELECT a.id,


        b.id,


        a.entity_id


 FROM   ec_session a,


        ec_session b


 WHERE  a.entity_type = 'jobStep'


        AND b.entity_type = 'jobStep'


        AND a.entity_id = b.entity_id


        AND a.id != b.id;

The query result would look something like this:

 ID ID ENTITY_ID


 ---------------------- ---------------------- ----------------------


 6226119 6225025 13494960


 6225025 6226119 13494960


 6230676 6231067 13502403


 6231067 6230676 13502403

Any number that appears in both columns is a duplicate and one of the two entries has to be removed. Pick one of the entries and use SQL to delete the duplicates. The following SQL code deletes the duplicates in the above example:
```
 DELETE FROM ec_session_auth WHERE session_id IN ( 6226119, 6231067 );


 DELETE FROM ec_session WHERE id IN ( 6226119, 6231067 );


 commit;
```

Explanation

There appears to be a race condition - more prevalent when CloudBees CD (CloudBees Flow) is restarted - that enters duplicate jobIds into the database.