Insufficient Java Heap Space for the Crawler or Solr
If you have very large objects, they may not index due to insufficient Java memory for the Crawler or for Solr.
In the Crawler log you will see an ERROR similar to this:
Problem—Insufficient Crawler Java memory
Problem—Insufficient Solr Java memory
Also, if you search for ERROR in the Solr log under
Contenta_home\encaps\Solr\example\logs you will find an error similar to:
Solution—Increase the Java heap space for the Crawler or Solr (or both)
If you are having a problem with insufficient Java memory for the Crawler, when you increase Java memory for the Crawler, the problem will likely migrate to Solr.
To increase Java heap space on Windows, you will use the panels below. You open these by double-clicking on the system tray icons located near the end of the task bar. You may need to click on the black triangle to access them.
Increase memory in increments of 128. The critical field is “Maximum memory pool.” A value of 1024 should be sufficient for very large objects.
In the Solr service panel you will see a setup that was recommended by Apache Solr. Memory is controlled in the Java Options field. Consult your Solr documentation if you need further help The memory is set to 512 MB as depicted in the Java Options field.
You will need to restart the services to make the new memory settings take effect. The Start and Stop buttons are on the General tab.
On Linux, you will need to change the Java command line option for memory java - Xmx1024m … in the script, Contenta_home/bin/Contenta.csh, which starts the Crawler and Solr daemons. The memory setting is expressed in megabytes. For example 1024 MB. You will need to restart the daemons.
To instruct the Crawler to try again to index those very large objects that earlier failed to index, set RETRY to true in AppData (see Editing AppData keys and values).