Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Elastic search 1.4.2 data path issue #9249

Closed
dobariya opened this issue Jan 12, 2015 · 6 comments · Fixed by #9283
Closed

Elastic search 1.4.2 data path issue #9249

dobariya opened this issue Jan 12, 2015 · 6 comments · Fixed by #9283
Assignees
@dakrone
Labels
>bug v1.3.8 v1.4.3 v1.5.0 v2.0.0-beta1

Comments

@dobariya
Copy link

I am using UNC path to store the data eg: \pc1\data.
This works perfect in version 1.3.4 but after I upgrade it to 1.3.7 or 1.4.2 it started give me error

Below is the error log

[2015-01-12 16:03:33,087][INFO ][node ] [master1] version[1.3.7], pid[7068], build[3042293/2014-12-16T13:59:32Z]
[2015-01-12 16:03:33,087][INFO ][node ] [master1] initializing ...
[2015-01-12 16:03:33,102][INFO ][plugins ] [master1] loaded [], sites []
[2015-01-12 16:03:36,249][INFO ][node ] [master1] initialized
[2015-01-12 16:03:36,249][INFO ][node ] [master1] starting ...
[2015-01-12 16:03:36,514][INFO ][transport ] [master1] bound_address {inet[/0:0:0:0:0:0:0:0:9300]}, publish_address {inet[/192.168.2.101:9300]}
[2015-01-12 16:03:36,795][INFO ][discovery ] [master1] local1/xBx3MDWZT--1dKhACi3KSA
[2015-01-12 16:03:39,844][INFO ][cluster.service ] [master1] new_master [master1][xBx3MDWZT--1dKhACi3KSA][dell13][inet[/192.168.2.101:9300]]{master=true}, reason: zen-disco-join (elected_as_master)
[2015-01-12 16:03:39,901][INFO ][http ] [master1] bound_address {inet[/0:0:0:0:0:0:0:0:9200]}, publish_address {inet[/192.168.2.101:9200]}
[2015-01-12 16:03:39,901][INFO ][node ] [master1] started
[2015-01-12 16:03:39,964][INFO ][gateway ] [master1] recovered [0] indices into cluster_state
[2015-01-12 16:04:09,878][DEBUG][action.admin.cluster.node.stats] [master1] failed to execute on node [xBx3MDWZT--1dKhACi3KSA]
java.lang.IllegalStateException: Free bytes [-1] cannot be less than 0 or greater than total bytes [-1]
at org.elasticsearch.cluster.DiskUsage.(DiskUsage.java:32)
at org.elasticsearch.cluster.InternalClusterInfoService$ClusterInfoUpdateJob$1.onResponse(InternalClusterInfoService.java:279)
at org.elasticsearch.cluster.InternalClusterInfoService$ClusterInfoUpdateJob$1.onResponse(InternalClusterInfoService.java:260)
at org.elasticsearch.action.support.nodes.TransportNodesOperationAction$AsyncAction.finishHim(TransportNodesOperationAction.java:221)
at org.elasticsearch.action.support.nodes.TransportNodesOperationAction$AsyncAction.onOperation(TransportNodesOperationAction.java:196)
at org.elasticsearch.action.support.nodes.TransportNodesOperationAction$AsyncAction.access$900(TransportNodesOperationAction.java:96)
at org.elasticsearch.action.support.nodes.TransportNodesOperationAction$AsyncAction$2.run(TransportNodesOperationAction.java:140)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
@dakrone
Copy link
Member

dakrone commented Jan 12, 2015

@dobariya thanks for reporting this, can you tell me what kind of filesystem \pc1\data is? Is it a samba share or local disk?

@dakrone dakrone self-assigned this Jan 12, 2015
@dakrone dakrone added the >bug label Jan 12, 2015
@dobariya
Copy link
Author

@dakrone thanks for quick response. we have windows environment ,file system is NTFS

@dakrone
Copy link
Member

dakrone commented Jan 12, 2015

@dobariya also, as a workaround in the meantime, if you are not using the disk-based shard allocation you can disable this by setting cluster.routing.allocation.disk.threshold_enabled to false and the disk usages will not be checked.

@dobariya
Copy link
Author

@dakrone Thanks for the quick response and this resolved my issue.
I have one more question regarding version upgrade.
I have indexed 2 million data using version 1.3.4 and now i want to grab the latest elasticsearch version.Can I directly upgrade it by just replacing the existing setup elasticsearch-1.3.4 folder to the latest version or do I have to configure other settings also?

@dakrone
Copy link
Member

dakrone commented Jan 12, 2015

Can I directly upgrade it by just replacing the existing setup elasticsearch-1.3.4 folder to the latest version or do I have to configure other settings also?

You should be able to upgrade doing this, however, it would be good to check the release notes for the versions you are skipping to make sure there aren't any breaking changes that will impact you, in particular, if you're upgrading to 1.4.2, you should check the 1.4.0-beta1 release notes which lists the breaking changes between 1.3.x and 1.4.x (it is a small list)

@dobariya
Copy link
Author

Thanks buddy.
can you please guide me in setting up my configuration.
I have 3 nodes
node1:master=true and data=true
node2:master=true and data=false
node3:master=false and data=false

am storing the data on only one single UNC path which i already mentioned the the above comments.All above nodes are pointed to this single UNC path

Please guide whether above nodes setting, all nodes pointing the one single UNC path is a good practice or not

@dakrone dakrone mentioned this issue Jan 13, 2015
Closed
@dakrone dakrone mentioned this issue Jan 13, 2015
Merged
dakrone added a commit to dakrone/elasticsearch that referenced this issue Jan 26, 2015
@dakrone
Relax restrictions on filesystem size reporting
537769c 
Apparently some filesystems such as ZFS and occasionally NTFS can report
filesystem usages that are negative, or above the maximum total size of
the filesystem. This relaxes the constraints on `DiskUsage` so that an
exception is not thrown.

If 0 is passed as the totalBytes, `.getFreeDiskAsPercentage()` will
always return 100.0% free (to ensure the disk threshold decider fails
open)

Fixes elastic#9249
Relates to elastic#9260
dakrone added a commit that referenced this issue Jan 26, 2015
@dakrone
Relax restrictions on filesystem size reporting
84eda0e 
Apparently some filesystems such as ZFS and occasionally NTFS can report
filesystem usages that are negative, or above the maximum total size of
the filesystem. This relaxes the constraints on `DiskUsage` so that an
exception is not thrown.

If 0 is passed as the totalBytes, `.getFreeDiskAsPercentage()` will
always return 100.0% free (to ensure the disk threshold decider fails
open)

Fixes #9249
Relates to #9260
dakrone added a commit that referenced this issue Jan 26, 2015
@dakrone
Relax restrictions on filesystem size reporting
c799e19 
Apparently some filesystems such as ZFS and occasionally NTFS can report
filesystem usages that are negative, or above the maximum total size of
the filesystem. This relaxes the constraints on `DiskUsage` so that an
exception is not thrown.

If 0 is passed as the totalBytes, `.getFreeDiskAsPercentage()` will
always return 100.0% free (to ensure the disk threshold decider fails
open)

Fixes #9249
Relates to #9260
dakrone added a commit that referenced this issue Jan 26, 2015
@dakrone
Relax restrictions on filesystem size reporting
c54b60c 
Apparently some filesystems such as ZFS and occasionally NTFS can report
filesystem usages that are negative, or above the maximum total size of
the filesystem. This relaxes the constraints on `DiskUsage` so that an
exception is not thrown.

If 0 is passed as the totalBytes, `.getFreeDiskAsPercentage()` will
always return 100.0% free (to ensure the disk threshold decider fails
open)

Fixes #9249
Relates to #9260

Conflicts:
	src/main/java/org/elasticsearch/cluster/DiskUsage.java
	src/test/java/org/elasticsearch/cluster/DiskUsageTests.java
mute pushed a commit to mute/elasticsearch that referenced this issue Jul 29, 2015
@dakrone
Relax restrictions on filesystem size reporting
f6e224c 
Apparently some filesystems such as ZFS and occasionally NTFS can report
filesystem usages that are negative, or above the maximum total size of
the filesystem. This relaxes the constraints on `DiskUsage` so that an
exception is not thrown.

If 0 is passed as the totalBytes, `.getFreeDiskAsPercentage()` will
always return 100.0% free (to ensure the disk threshold decider fails
open)

Fixes elastic#9249
Relates to elastic#9260
mute pushed a commit to mute/elasticsearch that referenced this issue Jul 29, 2015
@dakrone
Relax restrictions on filesystem size reporting
b591043 
Apparently some filesystems such as ZFS and occasionally NTFS can report
filesystem usages that are negative, or above the maximum total size of
the filesystem. This relaxes the constraints on `DiskUsage` so that an
exception is not thrown.

If 0 is passed as the totalBytes, `.getFreeDiskAsPercentage()` will
always return 100.0% free (to ensure the disk threshold decider fails
open)

Fixes elastic#9249
Relates to elastic#9260

Conflicts:
	src/main/java/org/elasticsearch/cluster/DiskUsage.java
	src/test/java/org/elasticsearch/cluster/DiskUsageTests.java
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@dakrone dakrone

Labels
>bug v1.3.8 v1.4.3 v1.5.0 v2.0.0-beta1
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Relax restrictions on filesystem size reporting in DiskUsage dakrone/elasticsearch
2 participants
@dakrone @dobariya