Solr instance auto restarting in solr cloud mode

[brijesh27]

Hi,

1. I am running solr cloud with 6 solr instances having version solr5.2.1 and 3 zookeeper (3.4.7)
2. I have 2 questions regarding solr cloud :

How can i run solr cloud on port 80 smooothly, i have tried to run it but when i restart it gives me some exception , after doing some work i found that " ps aux | grep solr" its showing me the - -DSTOP.PORT= -920 , -920 port is nothing actually it was picking from script -1000 on solr port

e.g. -Djetty.port=8983 -DSTOP.PORT=7983

anyone help me how to fix and start solr cloud on port 80.

I have found some weird problem, every morning betweem 3 to 3:30 am my solr instance restarted automatically even no traffic pointed on them, no indexing is running on them. solr : 16G RAM and 10G heap size. Zookeeper servers are fine.

solr.log :

INFO - 2016-05-31 03:38:07.584; [ ] org.eclipse.jetty.util.log.Log; Logging initialized @261ms
INFO - 2016-05-31 03:38:07.732; [ ] org.eclipse.jetty.server.Server; jetty-9.2.10.v20150310
WARN - 2016-05-31 03:38:07.750; [ ] org.eclipse.jetty.server.handler.RequestLogHandler; !RequestLog
INFO - 2016-05-31 03:38:07.753; [ ] org.eclipse.jetty.server.NCSARequestLog; Opened /mnt/vol1/solr-5.2.1/server/logs/request.log
INFO - 2016-05-31 03:38:07.755; [ ] org.eclipse.jetty.deploy.providers.ScanningAppProvider; Deployment monitor [file:/mnt/vol1/solr-5.2.1/server/contexts/] at interval 0
INFO - 2016-05-31 03:38:08.066; [ ] org.eclipse.jetty.webapp.StandardDescriptorProcessor; NO JSP Support for /solr, did not find org.apache.jasper.servlet.JspServlet
WARN - 2016-05-31 03:38:08.085; [ ] org.eclipse.jetty.security.ConstraintSecurityHandler; ServletContext@o.e.j.w.WebAppContext@55f3ddb1{/solr,file:/mnt/vol1/solr-5.2.1/server/solr-webapp/webapp/,STARTING}{/solr.war} has uncovered http methods for path: /
INFO - 2016-05-31 03:38:08.105; [ ] org.apache.solr.servlet.SolrDispatchFilter; SolrDispatchFilter.init()WebAppClassLoader=1047503754@3e6fa38a
INFO - 2016-05-31 03:38:08.117; [ ] org.apache.solr.core.SolrResourceLoader; JNDI not configured for solr (NoInitialContextEx)
INFO - 2016-05-31 03:38:08.117; [ ] org.apache.solr.core.SolrResourceLoader; using system property solr.solr.home: /home/test/solr/server/solr

gc log:
OpenJDK 64-Bit Server VM (25.77-b03) for linux-amd64 JRE (1.8.0_77-b03), built on Mar 25 2016 00:42:26 by "mockbuild" with gcc 4.4.7 20120313 (Red Hat 4.4.7-16)
Memory: 4k page, physical 16333628k(13548580k free), swap 2097148k(2097148k free)
CommandLine flags: -XX:CMSInitiatingOccupancyFraction=50 -XX:CMSMaxAbortablePrecleanTime=6000 -XX:+CMSParallelRemarkEnabled -XX:+CMSScavengeBeforeRemark -XX:ConcGCThreads=4 -XX:InitialHeapSize=10737418240 -XX:MaxHeapSize=10737418240 -XX:MaxTenuringThreshold=8 -XX:NewRatio=3 -XX:OldPLABSize=16 -XX:ParallelGCThreads=4 -XX:+ParallelRefProcEnabled -XX:PretenureSizeThreshold=67108864 -XX:+PrintGC -XX:+PrintGCApplicationStoppedTime -XX:+PrintGCDateStamps -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+PrintHeapAtGC -XX:+PrintTenuringDistribution -XX:SurvivorRatio=4 -XX:TargetSurvivorRatio=90 -XX:ThreadStackSize=256 -XX:+UseCMSInitiatingOccupancyOnly -XX:+UseCompressedClassPointers -XX:+UseCompressedOops -XX:+UseConcMarkSweepGC -XX:+UseParNewGC
2016-05-31T03:38:07.586+0530: 0.265: Total time for which application threads were stopped: 0.0002140 seconds, Stopping threads took: 0.0000460 seconds
2016-05-31T03:38:07.741+0530: 0.420: Total time for which application threads were stopped: 0.0001655 seconds, Stopping threads took: 0.0000257 seconds
2016-05-31T03:38:07.789+0530: 0.468: Total time for which application threads were stopped: 0.0002245 seconds, Stopping threads took: 0.0000462 seconds
2016-05-31T03:38:07.800+0530: 0.479: Total time for which application threads were stopped: 0.0002235 seconds, Stopping threads took: 0.0000525 seconds

[pdurbin]

@brijesh27 hi, are you running the Dataverse code based of pull request #2985 from @bionary ( https://github.com/bionary/dataverse/commits/2322-solrCloud )? I'm asking because until that pull request is merged we recommend running Solr 4.6.0. For the latest docs on installing Solr for Dataverse, please see http://guides.dataverse.org/en/4.3.1/installation/prerequisites.html#solr

[pdurbin]

I'm closing this issue but if @brijesh27 or others feel it should be re-opened, please go ahead. Again, see also pull request #2985.