Sense Monitor App Reload Failure

QlikSense

QlikSense Monitor App Reload Failure

Please perform the following:


1. First, it's very important to verify that the Service Account is added to your proper AD Group and is an admin, especially if your UDC in the QMC uses an LDAP filter.


2. If the user account to run the Qlik Sense Services has changed, follow these instructions. If not, please skip this step:



 http://help.qlik.com/en-US/sense/November2017/Subsystems/PlanningQlikSenseDeployments/Content/Deployment/Change-user-accounts.htm



3. Temporarily disable antivirus, and then open all ports in the firewall that are needed according to the only documentation for the November release.


4. Check that all services are running and can be restarted, as well as verifying the user account running them.


5. Now, let’s dive into the Admin Portal and have you open the QMC


6. Delete all connections in the "Data Connections" tab that begin with "QRS_" (before June 2017) and "Monitor_Apps_" (after June 2017) -- delete all of them for both with those names


7. Delete all monitoring apps in the "Apps" tab that are named License Monitor, Operations Monitor, etc.


8. Minimize QMC. Delete all "QVD" files within the Windows Directory that related to the monitoring apps (usually C:\ProgramData\Qlik\Sense\Log"). All files related to "reloads", "governance", "monitor", "etc... any QVD file listed in this Log folder needs to be deleted except ones with "central logging"


9. In addition, go to the shared ArchivedLog shared folder in Windows and ZIP all log files especially the Engine logs, as anything older than 6 months can sometimes cause the reloads to fail. Once a ZIP is done, delete all log files.


10. Open QMC


11. Check the Virtual Proxy to see if they use a prefix. If so, this prefix needs to be included in the DC connection strings.


12. Check the Proxy and verify the HTTPS Default port is 443. If not, update the DC connection strings.


13. Check the Proxy and verify that Kerberos Authentication is notenabled.


14. It is required that at least one virtual proxy uses the Windows authentication pattern for the monitoring apps to work. Please confirm you are using "Windows" in the Virtual Proxy settings under the "Authentication" section.


15. Now, in the Apps section of the QMC, import the new default monitoring apps using the "Import" button within the "Apps" section of the QMC. C:\ProgramData\Qlik\Sense\Repository\DefaultApps


16. After which, click on "Tasks" under "Associated Items" for each Monitoring App that was imported and create an automatic rule to have them automatically refresh. Set the time for the first start to 1 minute from now so that the trigger will automatically kick off once you save.


17. Go to the "Data Connections" section of the QMC to verify that new "QRS_" (before June 2017) or "Monitor_Apps_" (June 2017+) data connections were re-created.


18. Open one of them and view the entire connection string. Verify that this exists in the string ";skipServerCertificateValidation=true;"


19. Verify if https://localhost (June or newer: listed twice in each data connection and both must be updated; 3.2.5 or older: listed once) or the FQDN should be listed in each connection string and change if needed. FQDN is required for multi-nodes or cloud solutions.


20. Go to the Apps section, click on the monitoring apps that were imported, choose "More Actions" and click "Reload Now".


21. Go to the "Tasks" tab to see if they reloaded. Hit refresh in the top right of the QMC page.


22. If the task fails, open each Data Connection beginning with "QRS_" (before June 2017) or "Monitor_Apps_" (after June 2017), and put in the User ID and Password for the Service Account running the Qlik Services. This account must be an administrator with Full Control and be a part of the required AD Groups listed in the installation guide.


23. In November 2017 release there is a code related issue. If it’s an upgraded November 2017 (non-patch) release then use a copy of the September 2017 mon app QVF files, in the QMC import them, then replace the November mon app files. The Data Connections and QVD files in the folder may or may not need to be deleted; trial and error.


24. Reload tasks manually and collect the Script Reload Logs.

Applying an SSL Certificate to Qlik Sense

Qliksense 

Applying an SSL Certificate to Qlik Sense

---

Below we outline the process to apply an SSL Certificate to a Qlik Sense server. This is requires prior knowledge of server operating system basics and admin rights.

1. Launch the MMC

2. When the MMC opens go to File|Add/Remove Snap-in. 

3. Click on the Certificates snap-in on the left side list box and click the add button. 

4. Choose Computer account and click Next.

5. Leave Local computer selected and click Finish. 

6. Click OK to go back to the MMC. 

7. The Local computer certificate store opens Right click on the Personal folder and choose Import.

8. Because we have specified the location to import a certificate, the store location choice is greyed out. Click Next.

9. Browse to the location of the certificate containing private key (the pfx file) and click Next. 

10. Enter the password set during certificate merge process.

11. Prompted with the Certificate store to place the certificate, make sure the radio button is highlighted for Place all cert... and the store is Personal. Click Next.

12. Click Finish and the certificate will be imported. 

13. Double click on the imported certificate and choose details.

14. Navigate down the list and click on Thumbprint.

15. Copy the thumbprint with spaces from the textbox. 

16. Browse to the Qlik Sense QMC. 

17. Click on the Proxies menu item.

18. Choose the appropriate proxy to input the thumbprint to use the 3rd party certificate. Click edit to open the proxy settings. 

19. Click on the security option and enter the thumbprint (with spaces) in the textbox. Click Apply. The QMC will ask to restart. Please click the Restart QMC button. You may be required to refresh the browser as well.

Node show RIM as "Node is offline"

QlikSense

Node show RIM as "Node is offline"

The certificates are installed on the RIM and the qlik services on the RIM are running.


This is in the repository system trace log located at:
C:\ProgramData\Qlik\Sense\Log\Repository\Trace\*System_Repository.LOG

67 20170830T130400.104-0500 INFO qlikServerName System.Repository.Repository.Core.SystemInformation 8 8caa1faa-d256-4ee5-af88-0bec186abe6c DOMAIN\qlikSvc Exception encountered when fetching app migration hash, still trying. Failed attempts: 5 The remote name could not be resolved: 'qlikServerName.domain.com' at System.Net.HttpWebRequest.EndGetResponse(IAsyncResult asyncResult)↵↓ at Qlik.Sense.Common.Communication.REST.Client.WebClientEx.GetWebResponse(WebRequest request, IAsyncResult result)↵↓ at System.Net.WebClient.DownloadBitsResponseCallback(IAsyncResult result) 8caa1faa-d256-4ee5-af88-0bec186abe6c


The DNS is not resolving the central node FQDN (name resolution is not happening)


Cause

lack of DNS resolution in the local network


Resolution

- Network Admin to configure adapters/DNS for proper DNS resolution.
- Edit hosts file on all nodes to contain each node IP and FQDN of each node in the       environment.


NOTE: hosts file can be located at ... C:\Windows\System32\drivers\etc

No Hub or QMC access after upgrade to Sense November 2017 or higher - recursive script

QlikSense

No Hub or QMC access after upgrade to Sense November 2017 or higher - recursive script

This article uses the recursive (recurse) cleanup sql script. After upgrading to Qlik Sense November 2017 or February 2018, the administrator is unable to access the Hub and/or QMC.  This article also fixes database issues and soft deletes, which cause numerous other problems besides being unable to access the QMC or HUB after an upgrade. This script is necessary to be run if upgrading from an older version to 11.14.3 or newer.

An inspection of the System_Repository log located in C:\ProgramData\Qlik\Sense\Log\Repository\Trace shows an error message relating to being unable to remove soft deleted records.

Sample:
Fatal exception Connection must be opened.↵↓Connection must be opened.↵↓Failed to remove soft deleted records↵↓An exception was thrown while invoking the constructor 'Void .ctor()' on type 'DatabaseContext'


On Qlik Sense November 2017 or higher

ISSUE

Till June everything is working fine.
But direct upgrade from June2017 to April2018
Fails the Repository Service to update some table which violates foreign key constraint.
C:\ProgramData\Qlik\Sense\Log\Repository\Trace\QLIKSERVER1_System_Repository
Fatal exception update or delete on table "CompositeEvents" violates foreign key constraint "FK_CompositeEventTimeConstraints_CompositeEvents_ID" on table "CompositeEventTimeConstraints"↵↓Failed to execute query: 'DELETE FROM "CompositeEvents" WHERE "Deleted" = true'. Exception: update or delete on table "CompositeEvents" violates foreign key constraint


CAUSE

In November 2017 and February 2018, when the Qlik Sense Repository Service initializes, it attempts to clean up the underlying Repository Database. If there are records which have broken referential integrity then the Repository Service will fail to initialize. This is a consequence of a change in behavior in November 2017/February 2018 where the Repository no longer does soft deletes.
The script may be found here: help.qlik.com

RESOLUTION

• Stop all services but the Qlik Sense Repository Database
• If using a stand-alone PostgreSQL server to host Qlik Sense content, then stop all Qlik Sense Services
• Take a backup of the Qlik Sense Repository Database
• Sample command using cmd.exe: cd / & mkdir QSR & "C:\Program Files\Qlik\Sense\Repository\PostgreSQL\9.6\bin\pg_dump.exe" -h localhost -p 4432 -U postgres -b -F t -f "c:\QSR\QSR_backup_%date:~-4,4%%date:~-10,2%%date:~-7,2%.tar" QSR
• This assumes that:
• Qlik Sense is installed to C:\Program Files
• That the site is not using a stand-alone PostgreSQL server
• Make any necessary adjustments if neither of those assumptions are true
• Download the attached .sql file and place it into the same directory as pg_dump.exe
• Run the following command in cmd.exe (run them one at a time and hit Enter then continue with the next one):
• cd C:\"Program Files"\Qlik\Sense\Repository\PostgreSQL\9.6\bin
• SET PGCLIENTENCODING=utf-8
• chcp 65001
• psql.exe -h localhost -p 4432 -U postgres -d QSR -e -f recurse_cleanup.sql
• If Qlik Sense is installed to a non-default directory then make the necessary adjustments
• Start all Qlik Sense Services
• Validate that the error about Failed to remove soft deleted records is not present in the System_Repository log
• If this does not resolve the issue then do contact Qlik Support for a more customized query

Changing of Hostname after installation of Qliksense

Qliksense

Changing of Hostname after installation of Qliksense

The hostname in Qlik Sense is determined by the content of host.cfg, to change the hostname, you must change the value in this file. This will also result in all certificates being invalidated. They need to be recreated. 

In a single node site this is sufficient, in a multi-node system there are further steps that will be required.

Single Node site

Qlik Sense any version 

Single Node

1) Log in as the service account

2) Change hostname in Windows as usual.

3) Restart the server as required by Windows.

4) Stop all the Qlik Sense services, except for the Qlik Sense Repository Database

5) Use Microsoft Management Console(MMC) to delete all the Qlik Sense related certificates.

6) Delete %ProgramData%\Qlik\Sense\Repository\Exported Certificates\.Local Certificates

7) Make a copy of %ProgramData%\Qlik\Sense\Host.cfg and rename the copy to Host.cfg.old

8) Host.cfg contains the hostname encoded in base64. Generate this string for the new hostname using a site such as base64encode. 

9) Open Host.cfg and replace the content with the new string

10) Finalize:

• In Qlik Sense 3.2.5 or earlier, start all Qlik Sense services to regenerate certificates. 
• In Qlik Sense June 2017* or later follow the instructions below to regenerate certificates:

Instead of starting the Repository Service directly, run Windows Command Prompt as Administrator.

Execute  "C:\Program Files\Qlik\Sense\Repository\Repository.exe" -bootstrap -iscentral -restorehostname 

*This is required in order to re-create local Qlik certificates in newer versions of sense, as this is due to changes in this version that introduces the floating central node concept. 

Keep in mind: The central certificate is used to encrypt password strings being used in connectors, so changing this certificate means that you will need to recreate all connectors that include password information.

Multi-Node Site

There are two possible scenarios in a multi-node environment changing the hostname for a Rim node and for a Central node. Certificates are always generated by the central node and are based on hostname so changing hostnames here is more time consuming and will require redistribution and in some cases also the deletion and readding of node information.

Changing Rim Node Hostname

• On the Rim node:

1) Change hostname in Windows as usual.

2) Restart the server as required by Windows.

3) Stop all the Qlik Sense services

4) Use Microsoft Management Console(MMC) to delete all the Qlik Sense related certificates.

5) Delete %ProgramData%\Qlik\Sense\Repository\Exported Certificates\.Local Certificates

6) For backup purposes, make a copy of %ProgramData%\Qlik\Sense\Host.cfg and rename the copy to Host.cfg.old

7) Host.cfg contains the hostname encoded in base64. Generate this string for the new hostname using a site such as base64encode. 

8) Open Host.cfg and replace the content with the new string

9) Start up Qlik Sense, when it fails to find the certificates it will enter set up mode

10) Unfortunately it is not possible to adjust the hostname of a node in QMC, so a when a Rim node's hostname is changed you have to then delete and readd it to do this

• On the Central server:

1) Open Qlik Management Console (QMC)

2) Click Nodes

3) Select the node whose hostname has changed

4) Click Delete

5) Confirm your decision

6) Click Create New

7) Fill in the node details as appropriate

8) You will need to recreate and/or modify any rules that specifically named the old node

Changing Central Node Certificate

All certificates used by Qlik Sense are created and signed by the central node and are based on its hostname, changing the hostname on the central node breaks this chain of trust and thus all certificates on all nodes will need to be recreated and redistributed.

• On every rim node:

1) Stop all Qlik Sense services

2) Use Microsoft Management Console (MMC) to delete all the Qlik Sense related certificates

3) Start Qlik Sense services

4) Upon starting without certificates they will enter Setup mode, meaning they will be listening for new certificates

• On the central node:

1) Change hostname in Windows as usual.

2) Restart the server as required by Windows.

3) Stop all the Qlik Sense services, except for the Qlik Sense Repository Database

4) Use Microsoft Management Console(MMC) to delete all the Qlik Sense related certificates.

5) Delete %ProgramData%\Qlik\Sense\Repository\Exported Certificates\.Local Certificates

6) Copy %ProgramData%\Qlik\Sense\Host.cfg to Host.cfg.old

7) Host.cfg contains the hostname encoded in base64. Generate this string for the new hostname using a site like https://www.base64decode.org/

8) Open Host.cfg and replace the content with the new string

9) Finalize:

• In Qlik Sense 3.2.5 or earlier, start all Qlik Sense services to regenerate certificates. 
• In Qlik Sense June 2017* or later follow the instructions below to regenerate certificates:

Instead of starting the Repository Service directly, run Windows Command Prompt as Administrator.

Execute  "C:\Program Files\Qlik\Sense\Repository\Repository.exe" -bootstrap -iscentral -restorehostname 

1)Confirm the new certificates are created and QMC is accessible

2) In the QMC, click Nodes

3) Select each rim node and click Redistribute

4) Follow the instructions displayed

Keep in mind: The central certificate is used to encrypt password strings being used in connectors, so changing this certificate means that you will need to recreate all connectors that include password information.

In addition to the above, the relevant tables in the repository database that contain the host name information are:

LocalConfigs ( Hostname column )

ServerNodeConfigurations ( Host Name )

Note: Please make sure that the above table hostname column should contain the correct server FQDN ( network host name and not the individual machine host name ) this should be exactly the hostname that is in the host.config file

Upgrading Qlik Sense

Qliksense 

Upgradation Qlik Sense From one version to another version & Impact on System of Upgradation On Live Server

Upgrade

Necessary steps before Upgradation

Before starting with the Upgradation on the Prod. Server, you must have the backups of all the QVDs, QVFs, Extensions, Widgets, .CSS file, the hub.js file and if you           have used the am Combo Chart extension, then the File(amCombo.js) with you.(you can find the am Combo.js at this location-"D:\StaticContent\Extensions\amCombo"
 [We have installed Qlik sense in the D driive, the location of the js file might differ on the basis of the Drive you have used to install Qlik Sense])

Taking backups from time to time is really important, because if your upgradation fails due to some issues, then you have an option of making the Prod server Live with the previous version only and with all the backups, that wouldn`t take much of your efforts and time.

-------------------------------------------------------------------------------------------------------------------------

Backing up of your Repository

Use this command to get the backup of Repository Database
"D:\QlikServer\Repository\PostgreSQL\9.6\bin\pg_dump.exe" -h localhost -p 4432 -U postgres -b -F t -f "D:\Backup_Certificate_Repository\QSR_backup%mydate%.tar" QSR 

Do the changes for location according to your setup.


This code can be used to get the automated backup of Qliksense Repository database backup on daily basis.

Make a batch file using TXT  File and Schedule it using Task Scheduler

//Start//

SET PGPASSWORD=Put Your Password Here

SET mydate=%DATE:/=-%_ %TIME::=-%

@echo off %mydate%

"D:\QlikServer\Repository\PostgreSQL\9.6\bin\pg_dump.exe" -h localhost -p 4432 -U postgres -b -F t -f "D:\Backup_Certificate_Repository\QSR_backup%mydate%.tar" QSR

//End//

*here PGPASSWORD is your Postgre database password

--------------------------------------------------------------------------------------------------------------------------

Backing up of your Certificates

Click On:-

Certificate backup link process

--------------------------------------------------------------------------------------------------------------------------

Start with Upgradation

1 Download the Nov. Version (Patch 0) of Qlik Sense Enterprise on the Production Server.
2 Double click on installtion file to start the installation, you will see an Option on the screen to "Upgrade" the Qlik Sense Version. 
3 Click on the "Upgrade" Button, and the Upgradation would start. 
4 This might take 20-30 minutes to complete the upgradation.
5 When the Upgradation is complete, Open the Qlik Sense Hub and QMC to verify Upgradation.
6 You will notice all the QVFs in the Hub, Open your App and browse through it.
7 You will notice, that all the extensions and widgets are absent in your app.
8 Now, import all the used Extensions and Widgets one by one and check the functionality of the app time to time to note if any errors/ issues persists?
9 After all the widgets and extensions have been imported, browse through the app to see if the application is working fine?
10 If everything works out fine, then you have successfully Upgraded the Qlik Sense Version.

Noticed Changes After Upgradation

1 All the CSS changes in your application have been reverted, i.e all the CSS changes have been removed after the upgradation.

NOTE You cannot replace the new .CSS file the backup of the CSS file, because if you would compare the CSS files of the Sept. Version and the Nov Version, you would notice a lot of new changes in the Nov Version. So the advise here is to make all the CSS changes one by one Manually in the new .CSS file.

2 If you have made some changes in the hub.js file, like making the application open up in the same tab instead of a new tab everytime, then such changes would no longer exist in your hub.js file.

"Go to C:\Program Files\Qlik\Sense\Client\hub and make a backup copy of the file hub.js

Right click on notepad.exe and open as an Administrator 
(must be opened as an admin in order to same modifications; if not you will 
be asked to rename and save as a different file)

Open the original hub.js file and make the following modification
ctrl + f to find the words ""window.open"" in the line 
""openApp:function(a,b,c){var d;d=c?e.buildUrl(!0,""sense"",""app"",a,""noData"",""true""):
e.buildUrl(!0,""sense"",""app"",a),b?window.open(d):e.setUrl(d)},openItem:function(a,b)
{b?window.open(a):e.setUrl(a)""Remove ""b?window.open(d):"" and ""b?window.open(a):"" and 
save the file

The line should read ""openApp:function(a,b,c){var d;d=c?e.buildUrl
(!0,""sense"",""app"",a,""noData"",""true""):e.buildUrl(!0,""sense"",""app"",a),
e.setUrl(d)},openItem:function(a,b){e.setUrl(a)}"" 

after the modification
Restart the Sense services, clear the internet browser cache, and then test"


It is advised to make all the changes manually again. 


3 The js for the am combo extension wont work, if you have a published application and you login it through an outside authentication page. This happens, because the js script requires a header in its script, so that the authentication page could allow it to pass through it and thus could be shown in the published app.


"paths: {
        
""amcharts"": ""/header/extensions/amCombo/library/amcharts"",
        
""amcharts.serial"": ""/header/extensions/amCombo/library/serial"",
        
""amcharts.theme.dark"": ""/header/extensions/amCombo/library/dark"",
        
""amcharts.theme.black"": ""/header/extensions/amCombo/library/black"",
        ""amcharts.theme.chalk"": ""/header/extensions/amCombo/library/light"",
        ""amcharts.theme.light"": ""/header/extensions/amCombo/library/chalk"",
        //""amcharts.theme.patterns"": ""/header/extensions/amCombo/library/patterns"""


If this header is not provided in the js, then the Published App would throw an error of "Invalid Visualisation" in places of the am combo extension used and, whenever you would toggle to the sheet where this extension is used, a login pop would appear everytime.

QlikSense Multinode Installation

QlikSense

QlikSense Multinode Installation

Central node(Installation of Central node)
Prerequisites (Get Port No 4444,4432,4242, 4747, 4748,4242, 5050, 5151,4244,4248,443,80 opened bothways for central and RIM)

1 During installation of central node please keep in mind to keep the listner ON for 4432 port .(Keep note of all the Password that you Put for Repository/Listener/QLIK).Complete the Installation.Once Done,Check the QMC and Hub.


  RIM Node(Installation)
2 Start installation of the RIM NODE.

3 During installtion of RIM Cluster Node,Remember to opt the option for JOIN CLUSTER.
3.1 Put the Server name of Central Node when you are on the Page where it asks for database and Listner server during RIM NODE Installtion.Use the same Password used while         installing the Central NODE.
3.2 If it Accepts Its good and if it fails Check for the 4432 Port Listener(Try to Telnet 4432 Port ,If it is not open then get it opened.And if it is opened and Listner         is not able to listen then you might have forgotten to keep listner ON while installing Central Node.For such case where port is open and you fail to listen , Follow         below violet colored Steps.)
3.2.1 Stop the services and Do the changes in Repository Files(Pghba.conf and Postgresql.conf) as detailed in below.
3.2.2 Turn on the services and continue with the steps of Installation.Complete the Installation.
  Central Node after installtion of RIM Node

4 Once the Installation is complete.Open the Central NODE and GO to QMC.
5 Click on NODE tab on left hand side of QMC Panel and click on create new from below,Give name for the new node(Eg NODE1)
6 Once you name the node.Put the host name of RIM Node Server here. It will give a certificate with a password and a link to be opened on RIM Node server.(Link-(HTTP://LOCALHOST:4570/CERTIFICATESETUP))

  On RIM Node
7 On RIM Node,  paste the URL into a new browser window.

8 On the Install certificates page (in your browser), enter the password and click Submit. If successful, you see the Successfully licensed message.

  On Central Node
9 To verify that rim nodes are configured correctly, open the QMC, click Nodes and you can see the status of the nodes in your deployment.
10 On NODE tab for Node Purpose Select Both.
11 On Node Cofiguration don’t Make changes
12 On Service Activation,Add the services that you want to run on NODE.
13 Apply the Settings.

14      Go to Virtal Proxy on Central node cluster.Add Node in load balancing.
15      Add the IP and Server name of RIM Node in Whitelist.Apply the settings.
16   Open both the server QMC and HUB to check the Node Installation.

-----------------------------------------------------------------------------------------------------------------------

Changes in Pghba.conf and Postgresql.conf

• Changes in Postgresql.conf (Match these settings)

# CONNECTIONS AND AUTHENTICATION

#------

listen_addresses = *

port = 4432

max_connections = 600

# - Connection Settings -

#listen_addresses = 'localhost'  

# what IP address(es) to listen on;   

# comma-separated list of addresses;

# defaults to 'localhost'; use '*' for all

# (change requires restart)

#port = 5432    

# (change requires restart)

#max_connections = 100   

• Changes in Pghba.conf (Match these settings)

# Put your actual configuration here

# If you want to allow non-local connections, you need to add more

# "host" records.  In that case you will also need to make PostgreSQL

# listen on a non-local interface via the listen_addresses

# configuration parameter, or via the -i or -h command line switches.

@authcomment@

# TYPE  DATABASE        USER            ADDRESS                 METHOD

@remove-line-for-nolocal@

# "local" is for Unix domain socket connections only

@remove-line-for-nolocal@local   all             all                                     @authmethodlocal@

# IPv4 local connections:

host    all             all             127.0.0.1/32            @authmethodhost@

# IPv6 local connections:

host    all             all             ::1/128                 @authmethodhost@

# Allow replication connections from localhost, by a user with the

# replication privilege.

@remove-line-for-nolocal@

#local   replication     @default_username@                                @authmethodlocal@

#host    replication     @default_username@        127.0.0.1/32            @authmethodhost@

#host    replication     @default_username@        ::1/128                 @authmethodhost@

host    all             all             0.0.0.0/0            md5

Follow the steps to achieve the multinode site for Qliksense. You can add required no of nodes as per your requirements.

Continue Qliking.....