Package Check Flavors
", + sprintf("Last updated on %s.
", format(Sys.time(), usetz=TRUE)), + "Package Check Results for Package %s
", pkg, pkg), - sprintf("Last updated on %s.
", format(Sys.time(), usetz=TRUE)), - sprintf("| Test job | OS type | Install | Check | Rout.fail | Memtest |
|---|
Package Check Results for Package %s
", pkg, pkg), + sprintf("Last updated on %s.
", format(Sys.time(), usetz=TRUE)), + sprintf("
+# data.table
[](https://cran.r-project.org/web/checks/check_results_data.table.html)
@@ -8,7 +8,6 @@
[](https://codecov.io/github/Rdatatable/data.table?branch=master)
[](https://gitlab.com/Rdatatable/data.table/pipelines)
[](https://www.rdocumentation.org/trends)
-[](http://depsy.org/package/r/data.table)
[](https://gitlab.com/jangorecki/rdeps)
[](https://gitlab.com/jangorecki/rdeps)
[](https://gitlab.com/jangorecki/rdeps)
@@ -16,14 +15,6 @@
`data.table` provides a high-performance version of [base R](https://www.r-project.org/about.html)'s `data.frame` with syntax and feature enhancements for ease of use, convenience and programming speed.
----
-
-**30 January 2020-List-columns in data.table - Tyson Barrett, [rstudio::conf(2020L)](https://rstudio.com/conference/)** -
- ---- - ## Why `data.table`? * concise syntax: fast to type, fast to read @@ -38,7 +29,7 @@ List-columns in data.table - Tyson Barrett, [rstudio::conf(2020L)](https://rstud * fast and friendly delimited **file reader**: **[`?fread`](https://rdatatable.gitlab.io/data.table/reference/fread.html)**, see also [convenience features for _small_ data](https://github.com/Rdatatable/data.table/wiki/Convenience-features-of-fread) * fast and feature rich delimited **file writer**: **[`?fwrite`](https://rdatatable.gitlab.io/data.table/reference/fwrite.html)** * low-level **parallelism**: many common operations are internally parallelized to use multiple CPU threads -* fast and scalable **aggregations**; e.g. 100GB in RAM (see [benchmarks](https://github.com/Rdatatable/data.table/wiki/Benchmarks-%3A-Grouping) on up to **two billion rows**) +* fast and scalable aggregations; e.g. 100GB in RAM (see [benchmarks](https://h2oai.github.io/db-benchmark/) on up to **two billion rows**) * fast and feature rich joins: **ordered joins** (e.g. rolling forwards, backwards, nearest and limited staleness), **[overlapping range joins](https://github.com/Rdatatable/data.table/wiki/talks/EARL2014_OverlapRangeJoin_Arun.pdf)** (similar to `IRanges::findOverlaps`), **[non-equi joins](https://github.com/Rdatatable/data.table/wiki/talks/ArunSrinivasanUseR2016.pdf)** (i.e. joins using operators `>, >=, <, <=`), **aggregate on join** (`by=.EACHI`), **update on join** * fast add/update/delete columns **by reference** by group using no copies at all * fast and feature rich **reshaping** data: **[`?dcast`](https://rdatatable.gitlab.io/data.table/reference/dcast.data.table.html)** (_pivot/wider/spread_) and **[`?melt`](https://rdatatable.gitlab.io/data.table/reference/melt.data.table.html)** (_unpivot/longer/gather_) @@ -80,7 +71,7 @@ DT[Petal.Width > 1.0, mean(Petal.Length), by = Species] ### Getting started -* [Introduction to data.table](https://cloud.r-project.org/web/packages/data.table/vignettes/datatable-intro.html) vignette +* [Introduction to data.table](https://cran.r-project.org/package=data.table/vignettes/datatable-intro.html) vignette * [Getting started](https://github.com/Rdatatable/data.table/wiki/Getting-started) wiki page * [Examples](https://rdatatable.gitlab.io/data.table/reference/data.table.html#examples) produced by `example(data.table)` @@ -90,7 +81,7 @@ DT[Petal.Width > 1.0, mean(Petal.Length), by = Species] ## Community -`data.table` is widely used by the R community. It is being directly used by hundreds of CRAN and Bioconductor packages, and indirectly by thousands. It is one of the [top most starred](http://www.r-pkg.org/starred) R package on GitHub. If you need help, the `data.table` community is active on [StackOverflow](http://stackoverflow.com/questions/tagged/data.table). +`data.table` is widely used by the R community. It is being directly used by hundreds of CRAN and Bioconductor packages, and indirectly by thousands. It is one of the [top most starred](https://www.r-pkg.org/starred) R packages on GitHub, and was highly rated by the [Depsy project](http://depsy.org/package/r/data.table). If you need help, the `data.table` community is active on [StackOverflow](https://stackoverflow.com/questions/tagged/data.table). ### Stay up-to-date diff --git a/configure b/configure index e29430ad77..f2e98ec312 100755 --- a/configure +++ b/configure @@ -1,4 +1,18 @@ -#!/bin/sh +#!/usr/bin/env sh + +# Find R compilers +CC=`${R_HOME}/bin/R CMD config CC` +CFLAGS=`${R_HOME}/bin/R CMD config CFLAGS` +# compiler and flags to 'cc' file +echo "CC=${CC}" > inst/cc +echo "CFLAGS=${CFLAGS}" >> inst/cc + +# gcc compiler info to output #3291 +case $CC in gcc*) + GCCV=`${CC} -dumpfullversion -dumpversion` + echo "$CC $GCCV" +esac + # Let's keep this simple. If pkg-config is available, use it. Otherwise print # the helpful message to aid user if compilation does fail. Note 25 of R-exts: # "[pkg-config] is available on the machines used to produce the CRAN binary packages" @@ -6,6 +20,7 @@ # and R-exts note 24 now suggests 'checkbashisms' as we proposed. msg=0 +NOZLIB=1 # if pkg-config is not available then zlib will be disabled for higher chance of compilation success pkg-config --version >/dev/null 2>&1 if [ $? -ne 0 ]; then echo "*** pkg-config is not installed." @@ -16,10 +31,11 @@ else echo "*** pkg-config is installed but 'pkg-config --exists zlib' did not return 0." msg=1 else + NOZLIB=0 lib=`pkg-config --libs zlib` - expr "$lib" : ".*-lz$" >/dev/null + expr -- "$lib" : ".*-lz$" >/dev/null # -- for FreeBSD, #4652 if [ $? -ne 0 ]; then - expr "$lib" : ".*-lz " >/dev/null + expr -- "$lib" : ".*-lz " >/dev/null # would use \b in one expr but MacOS does not support \b if [ $? -ne 0 ]; then echo "*** pkg-config is installed and 'pkg-config --exists zlib' succeeds but" @@ -31,12 +47,13 @@ else fi if [ $msg -ne 0 ]; then - echo "*** Compilation will now be attempted and if it works you can ignore this message. However," - echo "*** if compilation fails, try 'locate zlib.h zconf.h' and ensure the zlib development library" - echo "*** is installed :" + echo "*** Compilation will now be attempted and if it works you can ignore this message. In" + echo "*** particular, this should be the case on Mac where zlib is built in or pkg-config" + echo "*** is not installed. However, if compilation fails, try 'locate zlib.h zconf.h' and" + echo "*** ensure the zlib development library is installed :" echo "*** deb: zlib1g-dev (Debian, Ubuntu, ...)" echo "*** rpm: zlib-devel (Fedora, EPEL, ...)" - echo "*** brew: zlib (OSX)" + echo "*** There is a zlib in brew for OSX but the built in zlib should work." echo "*** Note that zlib is required to compile R itself so you may find the advice in the R-admin" echo "*** guide helpful regarding zlib. On Debian/Ubuntu, zlib1g-dev is a dependency of r-base as" echo "*** shown by 'apt-cache showsrc r-base | grep ^Build-Depends | grep zlib', and therefore" @@ -45,27 +62,43 @@ if [ $msg -ne 0 ]; then echo "*** 1) 'pkg-config --exists zlib' succeeds (i.e. \$? -eq 0)" echo "*** 2) 'pkg-config --libs zlib' contains -lz" echo "*** Compilation will now be attempted ..." - exit 0 +else + version=`pkg-config --modversion zlib` + echo "zlib ${version} is available ok" fi -version=`pkg-config --modversion zlib` -echo "zlib ${version} is available ok" - -# Find R compilers -CC=`${R_HOME}/bin/R CMD config CC` -CFLAGS=`${R_HOME}/bin/R CMD config CFLAGS` - # Test if we have a OPENMP compatible compiler # Aside: ${SHLIB_OPENMP_CFLAGS} does not appear to be defined at this point according to Matt's testing on # Linux, and R CMD config SHLIB_OPENMP_CFLAGS also returns 'no information for variable'. That's not # inconsistent with R-exts$1.2.1.1, though, which states it's 'available for use in Makevars' (so not # necessarily here in configure). Hence use -fopenmp directly for this detection step. # printf not echo to pass checkbashisms w.r.t. to the \n -printf "#include
-
+ // 1: 1 2,4 # should be 1,3
+ // 2: 2 2,4
+ //
+ // This has been fixed for a decade but the solution has changed over time.
+ //
+ // We don't wish to inspect the j expression for these cases because there are so many; e.g. user defined functions.
+ // A special symbol does not need to appear in j for the problem to occur. Using a member of .SD is enough as the example above illustrates.
+ // Using R's own reference counting could invoke too many unnecessary copies because these specials are routinely referenced.
+ // Hence we mark these specials (SD, BY, I) here in dogroups and if j's value is being assigned to a list column, we check to
+ // see if any specials are present and copy them if so.
+ // This keeps the special logic in one place in one file here. Previously this copy was done by memrecycle in assign.c but then
+ // with PR#4164 started to copy input list columns too much. Hence PR#4655 in v1.13.2 moved that copy here just where it is needed.
+ // Currently the marker is negative truelength. These specials are protected by us here and before we release them
+ // we restore the true truelength for when R starts to use vector truelength.
+ const int n = length(x);
+ // use length() not LENGTH() because LENGTH() on NULL is segfault in R<3.5 where we still define USE_RINTERNALS
+ // (see data.table.h), and isNewList() is true for NULL
+ if (n==0)
+ return false;
+ if (isVectorAtomic(x))
+ return ALTREP(x) || TRUELENGTH(x)<0;
+ if (isNewList(x)) {
+ if (TRUELENGTH(x)<0)
+ return true; // test 2158
+ for (int i=0; i