1.4.0 update HA setup fail

Y

yaboc

Member
Nov 23, 2017
35
3
8
35
hi

i'm trying to update HA 2 node setup, i pull updates on both servers, then when i try to update pending database updates i get

Code: 
php artisan migrate:status

  Migration name ............................................................. Batch / Status  
  2019_12_14_000001_create_personal_access_tokens_table ............................. Pending  
  2022_02_01_074017_create_sessions_table ........................................... Pending  
  2022_02_27_165517_create_password_resets_table .................................... Pending  
  2022_03_10_065030_create_user_domain_permission_table ............................. Pending  
  2022_04_20_215315_create_messages_table ........................................... Pending  
  2022_06_10_173727_user_name_info .................................................. Pending  
  2022_06_10_191009_rename_username ................................................. Pending  
  2022_06_26_193813_create_nemerald_app_users_table ................................. Pending  
  2022_06_28_193133_create_archive_recording ........................................ Pending  
  2022_07_09_142022_add_key_in_archive .............................................. Pending  
  2022_07_13_151759_change_u_u_i_d .................................................. Pending  
  2022_07_31_182545_create_failed_jobs_table ........................................ Pending  
  2022_08_18_054043_rename_app_table ................................................ Pending  
  2022_08_19_054211_add_status_to_mobile_app_users_table ............................ Pending  
  2022_12_03_073359_create_domain_groups_table ...................................... Pending  
  2022_12_04_154854_create_domain_group_relations_table ............................. Pending  
  2022_12_04_214810_create_user_domain_group_permissions_table ...................... Pending  
  2022_12_13_020849_create_webhook_calls_table ...................................... Pending  
  2023_01_18_000038_create_fax_allowed_emails_table ................................. Pending  
  2023_01_18_000135_create_fax_allowed_domain_names_table ........................... Pending  
  2024_01_30_200000_add_two_factor_columns_to_users_table ........................... Pending  
  2024_04_04_204123_add_two_factor_cookies_column_to_users_table .................... Pending  
  2024_05_02_055503_add_reference_id_to_messages_table .............................. Pending  
  2024_05_14_054105_create_activity_log_table ....................................... Pending  
  2024_05_16_191010_add_expires_at_to_personal_access_tokens_table .................. Pending  
  2024_05_21_052842_modify_migrations_table_to_use_uuids ............................ Pending  
  2024_05_21_162557_add_domain_uuid_to_activity_log_table ........................... Pending  
  2024_06_24_180349_create_extension_advanced_settings_table ........................ Pending  
  2024_07_03_153944_create_mobile_app_password_reset_links_table .................... Pending  
  2024_08_01_195937_drop_foreign_keys_in_relations .................................. Pending  
  2024_08_02_032654_update_cdr_table_add_status_field_if_not_exists ................. Pending  
  2024_08_20_062352_convert_text_to_uuid_in_relations ............................... Pending  
  2024_10_07_072614_create_pro_features_table ....................................... Pending  
  2024_10_21_204103_add_call_flow_to_v_xml_cdr_table ................................ Pending  
  2024_11_19_220709_create_whitelisted_numbers_table ................................ Pending  
  2024_12_01_194602_add_description_to_whitelisted_numbers_table .................... Pending  
  2024_12_19_324567_add_domain_to_mobile_app_password_reset_links_table ............. Pending  
  2025_02_14_021947_add_exclude_from_stale_report_to_mobile_app_users_table ......... Pending  
  2025_02_28_070016_create_wakeup_calls_table ....................................... Pending  
  2025_03_11_053022_create_wakeup_auth_ext_table .................................... Pending  
  2025_03_27_043634_create_emergency_calls_table .................................... Pending  
  2025_03_27_043753_create_emergency_call_members_table ............................. Pending  
  2025_04_09_042232_create_emergency_call_emails_table .............................. Pending  
  2025_04_29_004308_create_payment_gateways_and_gateway_settings_tables ............. Pending  
  2025_05_01_144332_create_business_hours_table ..................................... Pending  
  2025_05_01_144335_create_business_hour_periods_table .............................. Pending  
  2025_05_01_144337_create_business_hour_exceptions_table ........................... Pending  
  2025_06_19_052520_add_media_to_messages_table ..................................... Pending  
  2025_06_20_104709_create_cloud_provisioning_table ................................. Pending  
  2025_07_09_160113_add_primary_key_to_device_cloud_provisioning .................... Pending  
  2025_07_18_043755_create_v_sms_messages_table ..................................... Pending  
  2025_07_18_043904_create_v_sms_destinations_table ................................. Pending  
  2025_07_24_051536_add_extension_uuid_to_v_users_table ............................. Pending  
  2025_08_06_053911_create_locations_table .......................................... Pending  
  2025_08_13_060033_create_locationables_table ...................................... Pending  
  2025_08_16_052308_add_serial_number_to_v_devices_table ............................ Pending  
  2025_08_18_214748_create_provisioning_templates ................................... Pending  
  2025_08_21_175400_add_device_template_uuid_to_v_devices_table ..................... Pending  
  2025_08_29_210825_create_hotel_rooms_table copy ................................... Pending  
  2025_08_29_212048_create_hotel_room_status_table .................................. Pending  
  2025_09_02_000001_create_hotel_housekeeping_definitions_table ..................... Pending  
  2025_09_18_184333_create_ceretax_transactions_table ............................... Pending  
  2025_09_19_034557_create_billing_products_table ................................... Pending  
  2025_09_22_225113_add_primary_key_to_payment_gateways ............................. Pending  
  2025_09_22_225915_add_primary_key_to_gateway_settings ............................. Pending  
  2025_10_01_053802_create_email_log ................................................ Pending  
  2025_10_01_182153_create_hotel_pending_actions .................................... Pending  
  2025_10_19_061650_create_call_transcription_providers_table ....................... Pending  
  2025_10_19_062447_call_create_transcription_policy_table .......................... Pending  
  2025_10_19_062950_call_create_transcription_provider_config_table ................. Pending  
  2025_10_28_181730_create_call_transcriptions_table ................................ Pending  
  2026_01_22_210215_create_device_keys_table ........................................ Pending  

php artisan migrate

                                                                                             
                                  APPLICATION IN PRODUCTION.                                  
                                                                                             

 ┌ Are you sure you want to run this command? ──────────────────┐
 │ Yes                                                          │
 └──────────────────────────────────────────────────────────────┘

   INFO  Running migrations.

  2019_12_14_000001_create_personal_access_tokens_table ............................ 3ms FAIL

   Illuminate\Database\QueryException

  SQLSTATE[42P07]: Duplicate table: 7 ERROR:  relation "personal_access_tokens" already exists (Connection: pgsql, SQL: create table "personal_access_tokens" ("id" uuid not null default uuid_generate_v4(), "tokenable_type" varchar(255) not null, "tokenable_id" uuid not null, "name" varchar(255) not null, "token" varchar(64) not null, "abilities" text null, "last_used_at" timestamp(0) without time zone null, "created_at" timestamp(0) without time zone null, "updated_at" timestamp(0) without time zone null))

  at vendor/laravel/framework/src/Illuminate/Database/Connection.php:822
    818▕                     $this->getName(), $query, $this->prepareBindings($bindings), $e
    819▕                 );
    820▕             }
    821▕
  ➜ 822▕             throw new QueryException(
    823▕                 $this->getName(), $query, $this->prepareBindings($bindings), $e
    824▕             );
    825▕         }
    826▕     }

      +9 vendor frames
  10  database/migrations/2019_12_14_000001_create_personal_access_tokens_table.php:26
      Illuminate\Support\Facades\Facade::__callStatic()

      +25 vendor frames
  36  artisan:37
      Illuminate\Foundation\Console\Kernel::handle()

Any ideas as to why this is happening and how to fix it?
Thanks
y
 
Last edited:
The fact that every table is pending indicates a problem with your system or cluster. The last update only added one pending table update. Since all of your tables are pending, it tries to run all migrations again, and, of course, it hits an issue since those tables already exist.

Check your migrations table

Code: 
select * from migrations ;

It's probably empty on one of your servers or both.
Here is an example of a healthy-looking table:


Code: 
                                 migration                                 | batch |                  id                 
---------------------------------------------------------------------------+-------+--------------------------------------
 2019_12_14_000001_create_personal_access_tokens_table                     |     1 | e94883ff-e19e-4579-9fa6-453d28fedb1c
 2022_02_01_074017_create_sessions_table                                   |     1 | eff88e5a-bbad-47b3-966d-749376d8d39a
 2022_02_27_165517_create_password_resets_table                            |     1 | a2dbc8b6-33da-42b2-bc02-f08ffea72351
 2022_03_10_065030_create_user_domain_permission_table                     |     1 | 359f3a7a-b703-4e0c-a37e-b4550a8961f5
 2022_04_20_215315_create_messages_table                                   |     1 | 6c8be4ed-d01c-4519-806a-b4d16f2b90a9
 2022_06_10_173727_user_name_info                                          |     1 | 852015de-73de-4090-98be-190af5e5cb18
 2022_06_10_191009_rename_username                                         |     1 | c02e0de7-d86a-4979-9c82-a4d366ff2b56
 2022_06_26_193813_create_nemerald_app_users_table                         |     1 | a78bfc6f-dff9-455c-87a8-53f284959abc
 2022_06_28_193133_create_archive_recording                                |     1 | 685aa39d-71a8-49bc-8c5d-603ac0c31c14
 2022_07_09_142022_add_key_in_archive                                      |     1 | b28002b3-5c0b-468f-9dce-afa6d9e176fc
 2022_07_13_151759_change_u_u_i_d                                          |     1 | f5cb365c-02ac-4aaf-a550-98cd9366abfb
 2022_07_31_182545_create_failed_jobs_table                                |     1 | 955a1f5b-c1dc-45ba-8687-3e9789b5db87
 2022_08_18_054043_rename_app_table                                        |     1 | 38cef20f-5349-46d4-89d6-6db0c91c9712
 2022_08_19_054211_add_status_to_mobile_app_users_table                    |     1 | f0b2f018-be1c-41b0-9c03-44bd4d9be4e2
 2022_12_03_073359_create_domain_groups_table                              |     1 | e5946d05-8e44-478e-bf90-50e7b9889025
 2022_12_04_154854_create_domain_group_relations_table                     |     1 | ab70d2bc-cfb0-4e4d-8524-c5b8ec093330
 2022_12_04_214810_create_user_domain_group_permissions_table              |     1 | fb0e22d3-a42e-402f-b912-4510931f4b42
 2022_12_13_020849_create_webhook_calls_table                              |     1 | 899aef34-1f8c-467a-a156-e8e3ed5763d4
 
  • Like
Reactions: yaboc
@pbxgeek

it shows empty on both :/. i'm not sure how that happened i've only done one upgrade prior to 1.4.0 and i there was no pending changes.

Code: 
fusionpbx=# select * from migrations ;
 migration | batch | id
-----------+-------+----
(0 rows)

Is there a way to recover/fix this or do i need to redo the cluster ?
It seems there's an earlier backup that could help ?

Code: 
var/backups/fspbx# ls -alh
total 147M
drwxr-xr-x 3 root root 4.0K Jan  6 20:33 .
drwxr-xr-x 3 root root 4.0K Jan 30 00:00 ..
-rw-r--r-- 1 root root 147M Jan  6 20:33 backup_2026-01-07.tgz
drwxr-xr-x 2 root root 4.0K Jan  6 20:33 postgresql
/var/backups/fspbx# cd postgresql/
/var/backups/fspbx/postgresql# ls
fusionpbx_pgsql_2026-01-07.sql

Thank you,
y
 
You can check the backup, but I suspect this didn't happen to you today. It probably happened some time ago, and you just found out today during the update.

This is untested, but here is what you can try. You can try creating all those missing records manually in the migration table and assign batch =1. This will dictate that all migrations have already run. Skip the last one so it actually adds the missing table from the last update.
 
  • Like
Reactions: yaboc
@pbxgeek

per your suggestions i created all records before pulling the files

Code: 
                                 migration                                 | batch |                  id               
---------------------------------------------------------------------------+-------+--------------------------------------
 2019_12_14_000001_create_personal_access_tokens_table                     |     1 | 568c9163-8522-42d2-b1bf-1f5ba5fda385
 2022_02_01_074017_create_sessions_table                                   |     1 | 123dcc48-1922-4945-8789-c128969592bc
 2022_02_27_165517_create_password_resets_table                            |     1 | 5b27130b-57b2-4a64-abfe-f2c9da692ba8
 ...
  2025_10_19_062447_call_create_transcription_policy_table                  |     1 | f78ce79d-50e3-41a8-8113-0dbd728c6a98
 2025_10_19_062950_call_create_transcription_provider_config_table         |     1 | ba1f6e3b-3b80-4928-ac46-30e5eed34110
 2025_10_28_181730_create_call_transcriptions_table                        |     1 | 554a2c5f-6c35-4110-9938-970f36d6635e
(71 rows)

i see pending migration after pulling new files and checking pending db migrations

Code: 
2025_10_19_062447_call_create_transcription_policy_table .......................... [1] Ran
  2025_10_19_062950_call_create_transcription_provider_config_table ................. [1] Ran
  2025_10_28_181730_create_call_transcriptions_table ................................ [1] Ran
  2026_01_22_210215_create_device_keys_table ........................................ Pending

but when i run it successfully on server A

Code: 
php artisan migrate

                                                                                            
                                  APPLICATION IN PRODUCTION.                                 
                                                                                            

 ┌ Are you sure you want to run this command? ──────────────────┐
 │ Yes                                                          │
 └──────────────────────────────────────────────────────────────┘

   INFO  Running migrations.

  2026_01_22_210215_create_device_keys_table ...................................... 11ms DONE

it created the record in database with batch '2'

Code: 
2025_10_28_181730_create_call_transcriptions_table                        |     1 | 554a2c5f-6c35-4110-9938-970f36d6635e
 2026_01_22_210215_create_device_keys_table                                |     2 | 4ab2468d-4864-42f7-8d41-58473e952591
(72 rows)

which is replicated/migrated to server 2
is this going to be an issue ? should i update batch to 1 on that record on server A which will replicate to B? i'm assuming future updates will increment batch # as well?
is it better to rebuild to avoid future update issues?
thank you in advance
 
Batch 2 is correct. It assigns the next number every time you run a migration. But there is now a proper way to run on the second server. It replicated the migration status but it didn't actually create the table. Read our doc on how to perform these correctly on the second server. There is also a step that refreshes the bidirectional replication for new tables.

www.fspbx.com

How to update | FS PBX – Open Source PBX Dashboard for VoIP Providers

Below steps apply to single server deployments.
www.fspbx.com www.fspbx.com

And to answer your question. I'm pretty sure you will be ok now that you got this far as long as you follow our guide to finish it.
 
  • Like
Reactions: yaboc
pbxgeek said:
Batch 2 is correct. It assigns the next number every time you run a migration. But there is now a proper way to run on the second server. It replicated the migration status but it didn't actually create the table. Read our doc on how to perform these correctly on the second server. There is also a step that refreshes the bidirectional replication for new tables.

www.fspbx.com

How to update | FS PBX – Open Source PBX Dashboard for VoIP Providers

Below steps apply to single server deployments.
www.fspbx.com www.fspbx.com

And to answer your question. I'm pretty sure you will be ok now that you got this far as long as you follow our guide to finish it.
Click to expand...
Thanks @pbxgeek .
i did run the
Code: 
php artisan migrate:delete-last-batch
php artisan migrate
php artisan db:refresh-subscriptions
on Server 2 afterwards so i have
Code: 
2025_10_28_181730_create_call_transcriptions_table                        |     1 | 554a2c5f-6c35-4110-9938-970f36d6635e
 2026_01_22_210215_create_device_keys_table                                |     2 | 4ab2468d-4864-42f7-8d41-58473e952591
on both servers!

How can I check if my 1.3.3 update went ok as far as table creation? Were there any tables pushed with that update? Im pretty sure I've started with version 1.3.2 so I had one update to 1.3.3 prior to moving to current 1.4.0.

thank you again @pbxgeek !!! Have a wonderful day!
 
Last edited:
Just in case you missed it you need to run this command on BOTH servers after migrations.

php artisan db:refresh-subscriptions
 
  • Like
Reactions: yaboc
You must log in or register to reply here.
Top Bottom