- Update to version 1.16.1
* Fixed a bug with the template-coordinate sort which caused
incorrect ordering when using threads, or processing large
files that don't fit completely in memory.
* Fixed a crash that occurred when trying to use samtools merge
in template-coordinate mode.
- Update to version 1.16.0
* New work and changes:
- samtools reference command added. This subcommand extracts
the embedded reference out of a CRAM file.
- samtools import now adds grouped by query-name to the header
- Made samtools view read error messages more generic. Former
error message would claim that there was a "truncated file
or corrupt BAM index file" with no real justification. Also
reset errno in stream_view which could lead to confusing
error messages.
- Make samtools view -p also clear mqual, tlen and cigar.
- Add bedcov option -c to report read count.
- Add UMI/barcode handling to samtools markdup.
- Add a new template coordinate sort order to samtools sort
and samtools merge. This is useful when working with unique
molecular identifiers (UMIs).
- Rename mpileup --ignore-overlaps to
--ignore-overlaps-removal or --disable-overlap-removal.
- The dict command can now read BWA's .alt file and add AH:*
tags indicating reference sequences that represent
alternate loci.
- The samtools index command can now accept multiple alignment
filenames with the new -M option, and will index each of
them separately.
- Allow samtools fastq -T "*". This allows all tags from SAM
records to be written to fastq headers.
* Bug Fixes:
- Re-enable --reference option for samtools depth. The
reference is not used but this makes the command line
usage compatible with older releases.
- Fix regex coordinate bug in samtools markdup.
- Fix divide by zero in plot-bamstats -m, on unmapped data.
- Fix missing RG headers when using samtools merge -r.
- Fix a possible unaligned access in samtools reference.
* Documentation:
- Add documentation on CRAM compression profiles and some
of the newer options that appear in CRAM 3.1 and above.
- Add sclen filter expression keyword documentation.
- Extend FILTER EXPRESSION man page section to match the
changes made in HTSlib.
OBS-URL: https://build.opensuse.org/request/show/1003650
OBS-URL: https://build.opensuse.org/package/show/science/samtools?expand=0&rev=15
+ Changes affecting the whole of samtools, or multiple sub-commands:
* Samtools now uses the new HTSlib header API. As this adds more checks
for invalid headers, it is possible that some illegal files will now
be rejected when they would have been allowed by earlier versions.
Examples of problems that will now be rejected include @SQ lines with
no SN: tag, and @RG or @PG lines with no ID: tag.
* samtools sub-commands will now add '@PG' header lines to output sam/bam/cram
files. To disable this, use the '--no-PG' option.
* samtools now supports alignment records with reference positions greater
than 2 gigabases. This allows samtools to process alignments for
species which have large chromosomes, like axolotl and lungfish. Note that
due to file format limitations, data with large reference positions
must use the SAM format.
* Improved the efficiency of reading and writing SAM format data by 2 fold
(single thread). This is further improved by the ability to use multiple
threads, as previously done with BAM and CRAM.
* samtools can now write BGZF-compressed SAM format. To enable this,
either save files with a '.sam.gz' suffix, or use '--output-fmt sam.gz'.
* samtools can now index BGZF-compressed SAM files.
* The region parsing code has been improved to handle colons in reference
names. Strings can be disambiguated by the use of braces, so for
example when reference sequences called "chr1" and "chr1:100-200"
are both present, the regions "{chr1}:100-200" and "{chr1:100-200}"
unambiguously indicate which reference is being used.
* samtools flags, flagstats, idxstats and stats now have aliases
flag, flagstat, idxstat and stat.
* A new global '--write-index' option has been added. This allows output
sam.gz/bam/cram files to be indexed while they are being written out.
This should work with addreplacerg, depad, markdup, merge, sort, split,
and view.
* A global '--verbosity' option has been added to enable/disable
debugging output.
* It is now possible to have data and index files stored in different
locations.
* HTSlib (and therefore SAMtools) now uses version 4 signatures by default
for its s3:// plug-in. It can also write to S3 buckets, as long as
version 4 signatures are in use. See HTSlib's NEWS file and
htslib-s3-plugin manual page for more information.
* HTSlib (and therefore SAMtools) no longer considers a zero-length
file to be a valid SAM file. This has been changed so that pipelines such
as `somecmd | samtools ...` with `somecmd` aborting before outputting
anything will now propagate the error to the second command.
* The samtools manual page has been split up into one for each
sub-command. The main samtools.1 manual page now lists the sub-commands
and describes the common global options.
* The meaning of decode_md, store_md and store_nm in the fmt-option section
of the samtools.1 man page has been clarified.
* Fixed numerous memory leaks.
* Fixed incorrect macro definition on Windows.
* bedcov, phase, misc/ace2sam and misc/wgsim now check for failure to open
files.
+ Changes affecting specific sub-commands:
* A new "coverage" sub-command has been added. This prints a tabular format
of the average coverage and percent coverage for each reference sequence,
as well as number of aligned reads, average mapping quality and base
quality. It can also (with the '-m' option) plot a histogram of
coverage across the genome.
* samtools calmd:
> Reference bases in MD: tags are now converted to upper case.
* samtools depth:
> Add new options to write a header to the output (-H) and to direct
the output to a file (-o).
> New options '-g' and '-G' can be used to filter reads.
> Fix memory leak when failing to set CRAM options.
> Fix bug when using region filters where the '-a' option did not
work for regions with no coverage.
* samtools fasta and fastq:
> '-1 FILE -2 FILE' with the same filename now works properly.
> '-o FILE' is added as a synonym for '-1 FILE -2 FILE'.
> The '-F' option now defaults to 0x900 (SECONDARY,SUPPLEMENTARY).
Previously secondary and supplementary records were filtered internally
in a way that could not be turned off.
> Allow reading from a pipe without an explicit '-' on the command line.
> Turn on multi-threading for bgzf compressed output files.
> Fixed bug where the samtools fastq -i would output incorrect information
in the Casava tags for dual-index reads. It also now prints the tags
for dual indices in the same way as bcl2fastq, using a '+' sign between
the two parts of the index.
* samtools flagstat:
> Samtools flagstat can now optionally write its output in JSON format or
as a tab-separated values file..
* samtools markdup:
> It can optionally tag optical duplicates (reads following Illumina
naming conventions only). The is enabled with the '-d' option,
which sets the distance for duplicates to be considered as optical.
> The report stats (-s) option now outputs counts for optical and
non-primary (supplementary / secondary) duplicates. It also reports
the Picard "estimate library size" statistic. A new '-f' option
can be used to save the statistics in a given file.
> The rules for calling duplicates can be changed using the new --mode
option. This mainly changes the position associated with each read in
a pair. '--mode t' (the default) is the existing behaviour where the
position used is that of the outermost template base associated with the
read. Alternatively '--mode s' always uses the first unclipped sequence
base. In practice, this only makes a difference for read pairs where the
two reads are aligned in the same direction.
> A new '-c' option can be used to clear any existing duplicate tags.
> A new '--include-fails' option makes markdup include QC-failed reads.
> Fixed buffer overflow in temporary file writer when writing a mixture
of long and short alignment records.
* samtools mpileup:
> mpileup can now process alignments including CIGAR P (pad) operators
correctly. They will now also produce the correct output for alignments
where insertions are immediately followed by deletions, or deletions by
insertions. Note that due to limitations in HTSlib, they are still
unable to output sequences that have been inserted before the first
aligned base of a read..
> In samtools mpileup, a deletion or pad on the reverse strand is now
marked with a different character ('#') than the one used on a forward
strand ('*'), if the '--reverse-del' option is used.
> New option '--output-extra' can be used to add columns for user
selected alignment fields or aux tags.
> Fixed double-counting of overlapping bases in alignment records with
deletions or reference skips longer than twice the insert size.
> Improved manual page with documentation about what each output column
means.
* samtools quickcheck:
> Add unmapped (-u) option, which disables the check for @SQ lines in
the header.
* samtools reheader:
> A new option '-c' allows the input header to be passed to a given
command. Samtools then takes the output of this command and uses it
as the replacement header.
> Make it clear in help message that reheader --in-place only works on
CRAM files.
> Refuse to in-place reheader BAM files, instead of unexpectedly writing
a BAM file to stdout.
* samtools split:
> In samtools split, the '-u' option no longer accepts an extra file name
from which a replacement header was read. The two file names were
separated using a colon, which caused problems on Windows and prevented
the use of URLs. A new '-h' option has been added to allow the replacement
header file to be specified in its own option.
> Fixed bug where samtools split would crash if it read a SAM header that
contained an @RG line with no ID tag.
* samtools stats:
> stats will now compute base compositions for BC, CR, OX and RX tags,
and quality histograms for QT, CY, BZ and QX tags.
> New stats FTC and LTC showing total number of nucleotides for first and
last fragments.
> The rules for classifying reads as "first" or "last" fragment have been
tightened up.
> Fixed bug where stats could over-estimate coverage when using the
target-regions option or when a region was specified on the command-line..
> Fixed error in stats GCD percentile depth calculation when the depth to be
reported fell between two bins. It would report the depth entirely from
the lower bin instead of taking a weighted average of the two.
> Better catching and reporting of out of memory conditions.
> Improved manual page.
* samtools tview:
> tview can now display alignments including CIGAR P operators, D followed
by I and I followed by D correctly. See mpileup above for more
information.
> The "go to position" text entry box has been made wider.
> Fixed samtools tview -s option which was not filtering reads correctly.
It now only shows reads from the requested sample or read group.
* samtools view:
> New options '-d' and '-D' to only output alignments which have a tag
with a given type and value.
* misc/plot-bamstats script:
> Fixed merge (-m) option.
> Made the quality heatmap work with gnuplot version 5.2.7 and later.
> Fixed --do-ref-stats bug where fasta header lines would be counted
as part of the sequence when the --targets option was used.
* Removed the misc/varfilter.py Python script, as it takes consensus-pileup
as input, which was removed from samtools in release 0.1.17 in 2011.
OBS-URL: https://build.opensuse.org/request/show/770319
OBS-URL: https://build.opensuse.org/package/show/science/samtools?expand=0&rev=6
- Update to 1.9
* Samtools mpileup VCF and BCF output is now deprecated. It is still
functional, but will warn. Please use bcftools mpileup instead. (#884)
* Samtools mpileup now handles the '-d' max_depth option differently. There
is no longer an enforced minimum, and '-d 0' is interpreted as limitless
(no maximum - warning this may be slow). The default per-file depth is
now 8000, which matches the value mpileup used to use when processing
a single sample. To get the previous default behaviour use the higher
of 8000 divided by the number of samples across all input files, or 250.
(#859)
* Samtools stats new features:
- The '--remove-overlaps' option discounts overlapping portions of
templates when computing coverage and mapped base counting. (#855)
- When a target file is in use, the number of bases inside the
target is printed and the percentage of target bases with coverage
above a given threshold specified by the '--cov-threshold' option. (#855)
- Split base composition and length statistics by first and last reads.
(#814, #816)
* Samtools faidx new features:
- Now takes long options. (#509, thanks to Pierre Lindenbaum)
- Now warns about zero-length and truncated sequences due to the
requested range being beyond the end of the sequence. (#834)
- Gets a new option (--continue) that allows it to carry on
when a requested sequence was not in the index. (#834)
- It is now possible to supply the list of regions to output in a text
file using the new '--region-file' option. (#840)
- New '-i' option to make faidx return the reverse complement of
the regions requested. (#878)
- faidx now works on FASTQ (returning FASTA) and added a new
fqidx command to index and return FASTQ. (#852)
* Samtools collate now has a fast option '-f' that only operates on
primary pairs, dropping secondary and supplementary. It tries to write
pairs to the final output file as soon as both reads have been found. (#818)
* Samtools bedcov gets a new '-j' option to make it ignore deletions (D) and
reference skips (N) when computing coverage. (#843)
* Small speed up to samtools coordinate sort, by converting it to use
radix sort. (#835, thanks to Zhuravleva Aleksandra)
* Samtools idxstats now works on SAM and CRAM files, however this
isn't fast due to some information lacking from indices. (#832)
* Compression levels may now be specified with the level=N
output-fmt-option. E.g. with -O bam,level=3.
* Various documentation improvements.
* Bug-fixes:
- Improved error reporting in several places. (#827, #834, #877, cd7197)
- Various test improvements.
- Fixed failures in the multi-region iterator (view -M) when regions
provided via BED files include overlaps (#819, reported by Dave Larson).
- Samtools stats now counts '=' and 'X' CIGAR operators when
counting mapped bases. (#855)
- Samtools stats has fixes for insert size filtering (-m, -i). (#845; #697
reported by Soumitra Pal)
- Samtools stats -F now longer negates an earlier -d option. (#830)
- Fix samtools stats crash when using a target region. (#875, reported by
John Marshall)
- Samtools sort now keeps to a single thread when the -@ option is absent.
Previously it would spawn a writer thread, which could cause the CPU
usage to go slightly over 100%. (#833, reported by Matthias Bernt)
- Fixed samtools phase '-A' option which was incorrectly defined to take
a parameter. (#850; #846 reported by Dianne Velasco)
- Fixed compilation problems when using C_INCLUDE_PATH. (#870; #817 reported
by Robert Boissy)
- Fixed --version when built from a Git repository. (#844, thanks to John
Marshall)
- Use noenhanced mode for title in plot-bamstats. Prevents unwanted
interpretation of characters like underscore in gnuplot version 5. (#829,
thanks to M. Zapukhlyak)
- blast2sam.pl now reports perfect match hits (no indels or mismatches).
(#873, thanks to Nils Homer)
- Fixed bug in fasta and fastq subcommands where stdout would not be flushed
correctly if the -0 option was used.
- Fixed invalid memory access in mpileup and depth on alignment records
where the sequence is absent.
OBS-URL: https://build.opensuse.org/request/show/633581
OBS-URL: https://build.opensuse.org/package/show/science/samtools?expand=0&rev=5
