Package org.roaringbitmap.buffer

Class MutableRoaringBitmap

  • All Implemented Interfaces:
    java.io.Externalizable, java.io.Serializable, java.lang.Cloneable, java.lang.Iterable<java.lang.Integer>, AppendableStorage<MappeableContainer>, BitmapDataProvider, ImmutableBitmapDataProvider
    public class MutableRoaringBitmap
extends ImmutableRoaringBitmap
implements java.lang.Cloneable, java.io.Serializable, java.lang.Iterable<java.lang.Integer>, java.io.Externalizable, BitmapDataProvider, AppendableStorage<MappeableContainer>
    MutableRoaringBitmap, a compressed alternative to the BitSet. It is similar to org.roaringbitmap.RoaringBitmap, but it differs in that it can interact with ImmutableRoaringBitmap objects in the sense that MutableRoaringBitmap is derived from MutableRoaringBitmap. A MutableRoaringBitmap is an instance of an ImmutableRoaringBitmap (where methods like "serialize" are implemented). That is, they both share the same core (immutable) methods, but a MutableRoaringBitmap adds methods that allow you to modify the object. This design allows us to use MutableRoaringBitmap as ImmutableRoaringBitmap instances when needed. MutableRoaringBitmap instances can be casted to an ImmutableRoaringBitmap instance in constant time which means that code written for ImmutableRoaringBitmap instances run at full speed (without copies) on MutableRoaringBitmap instances. A MutableRoaringBitmap can be used much like an org.roaringbitmap.RoaringBitmap instance, and they serialize to the same output. The RoaringBitmap instance will be faster since it does not carry the overhead of a ByteBuffer back-end, but the MutableRoaringBitmap can be used as an ImmutableRoaringBitmap instance. Thus, if you use ImmutableRoaringBitmap, you probably need to use MutableRoaringBitmap instances as well; if you do not use ImmutableRoaringBitmap, you probably want to use only RoaringBitmap instances. 
     
      import org.roaringbitmap.buffer.*;

      //...

      MutableRoaringBitmap rr = MutableRoaringBitmap.bitmapOf(1,2,3,1000);
      MutableRoaringBitmap rr2 = new MutableRoaringBitmap();
      for(int k = 4000; k<4255;++k) rr2.add(k);

      RoaringBitmap rror = RoaringBitmap.or(rr, rr2);

      //...
      DataOutputStream wheretoserialize = ...
      rr.runOptimize(); // can help compression
      rr.serialize(wheretoserialize);
    Integers are added in unsigned sorted order. That is, they are treated as unsigned integers (see Java 8's Integer.toUnsignedLong function). Up to 4294967296 integers can be stored.
    See Also:
    ImmutableRoaringBitmap, RoaringBitmap, Serialized Form

    • Constructor Detail

      • MutableRoaringBitmap

        public MutableRoaringBitmap()
        Create an empty bitmap

      • MutableRoaringBitmap

        public MutableRoaringBitmap​(RoaringBitmap rb)
        Create a MutableRoaringBitmap from a RoaringBitmap. The RoaringBitmap is not modified.
        Parameters:
        rb - the original bitmap

    • Method Detail

      • addOffset

        public static MutableRoaringBitmap addOffset​(ImmutableRoaringBitmap x,
                                             long offset)
        Generate a new bitmap, but with all its values incremented by offset. The parameter offset can be negative. Values that would fall outside of the valid 32-bit range are discarded so that the result can have lower cardinality.
        Parameters:
        x - source bitmap
        offset - increment (can be negative)
        Returns:
        a new bitmap

      • add

        public static MutableRoaringBitmap add​(MutableRoaringBitmap rb,
                                       long rangeStart,
                                       long rangeEnd)
        Generate a new bitmap with all integers in [rangeStart,rangeEnd) added.
        Parameters:
        rb - initial bitmap (will not be modified)
        rangeStart - inclusive beginning of range
        rangeEnd - exclusive ending of range
        Returns:
        new bitmap

      • add

        @Deprecated
public static MutableRoaringBitmap add​(MutableRoaringBitmap rb,
                                       int rangeStart,
                                       int rangeEnd)
        Deprecated.
        use the version where longs specify the range
        Generate a new bitmap with all integers in [rangeStart,rangeEnd) added.
        Parameters:
        rb - initial bitmap (will not be modified)
        rangeStart - inclusive beginning of range
        rangeEnd - exclusive ending of range
        Returns:
        new bitmap

      • and

        public static MutableRoaringBitmap and​(MutableRoaringBitmap x1,
                                       MutableRoaringBitmap x2)
        Bitwise AND (intersection) operation. The provided bitmaps are *not* modified. This operation is thread-safe as long as the provided bitmaps remain unchanged.
        Parameters:
        x1 - first bitmap
        x2 - other bitmap
        Returns:
        result of the operation

      • andNot

        public static MutableRoaringBitmap andNot​(MutableRoaringBitmap x1,
                                          MutableRoaringBitmap x2)
        Bitwise ANDNOT (difference) operation. The provided bitmaps are *not* modified. This operation is thread-safe as long as the provided bitmaps remain unchanged.
        Parameters:
        x1 - first bitmap
        x2 - other bitmap
        Returns:
        result of the operation

      • add

        public void add​(int... dat)
        Set all the specified values to true. This can be expected to be slightly faster than calling "add" repeatedly. The provided integers values don't have to be in sorted order, but it may be preferable to sort them from a performance point of view.
        Parameters:
        dat - set values

      • addN

        public void addN​(int[] dat,
                 int offset,
                 int n)
        Set the specified values to true, within given boundaries. This can be expected to be slightly faster than calling "add" repeatedly on the values dat[offset], dat[offset+1],..., dat[offset+n-1]. The provided integers values don't have to be in sorted order, but it may be preferable to sort them from a performance point of view.
        Parameters:
        dat - set values
        offset - from which index the values should be set to true
        n - how many values should be set to true

      • bitmapOf

        public static MutableRoaringBitmap bitmapOf​(int... dat)
        Generate a bitmap with the specified values set to true. The provided integers values don't have to be in sorted order, but it may be preferable to sort them from a performance point of view.
        Parameters:
        dat - set values
        Returns:
        a new bitmap

      • rangeSanityCheck

        protected static void rangeSanityCheck​(long rangeStart,
                                       long rangeEnd)

      • flip

        public static MutableRoaringBitmap flip​(MutableRoaringBitmap bm,
                                        long rangeStart,
                                        long rangeEnd)
        Complements the bits in the given range, from rangeStart (inclusive) rangeEnd (exclusive). The given bitmap is unchanged.
        Parameters:
        bm - bitmap being negated
        rangeStart - inclusive beginning of range
        rangeEnd - exclusive ending of range
        Returns:
        a new Bitmap

      • flip

        @Deprecated
public static MutableRoaringBitmap flip​(MutableRoaringBitmap rb,
                                        int rangeStart,
                                        int rangeEnd)
        Deprecated.
        use the version where longs specify the range
        Complements the bits in the given range, from rangeStart (inclusive) rangeEnd (exclusive). The given bitmap is unchanged.
        Parameters:
        rb - bitmap being negated
        rangeStart - inclusive beginning of range
        rangeEnd - exclusive ending of range
        Returns:
        a new Bitmap

      • or

        public static MutableRoaringBitmap or​(MutableRoaringBitmap x1,
                                      MutableRoaringBitmap x2)
        Bitwise OR (union) operation. The provided bitmaps are *not* modified. This operation is thread-safe as long as the provided bitmaps remain unchanged.
        Parameters:
        x1 - first bitmap
        x2 - other bitmap
        Returns:
        result of the operation

      • remove

        public static MutableRoaringBitmap remove​(MutableRoaringBitmap rb,
                                          long rangeStart,
                                          long rangeEnd)
        Generate a new bitmap with all integers in [rangeStart,rangeEnd) removed.
        Parameters:
        rb - initial bitmap (will not be modified)
        rangeStart - inclusive beginning of range
        rangeEnd - exclusive ending of range
        Returns:
        new bitmap

      • remove

        @Deprecated
public static MutableRoaringBitmap remove​(MutableRoaringBitmap rb,
                                          int rangeStart,
                                          int rangeEnd)
        Deprecated.
        use the version where longs specify the range
        Generate a new bitmap with all integers in [rangeStart,rangeEnd) removed.
        Parameters:
        rb - initial bitmap (will not be modified)
        rangeStart - inclusive beginning of range
        rangeEnd - exclusive ending of range
        Returns:
        new bitmap

      • xor

        public static MutableRoaringBitmap xor​(MutableRoaringBitmap x1,
                                       MutableRoaringBitmap x2)
        Bitwise XOR (symmetric difference) operation. The provided bitmaps are *not* modified. This operation is thread-safe as long as the provided bitmaps remain unchanged.
        Parameters:
        x1 - first bitmap
        x2 - other bitmap
        Returns:
        result of the operation

      • add

        public void add​(int x)
        Add the value to the container (set the value to "true"), whether it already appears or not. Java lacks native unsigned integers but the x argument is considered to be unsigned. Within bitmaps, numbers are ordered according to Integer.compareUnsigned(int, int). We order the numbers like 0, 1, ..., 2147483647, -2147483648, -2147483647,..., -1.
        Specified by:
        add in interface BitmapDataProvider
        Parameters:
        x - integer value

      • add

        public void add​(long rangeStart,
                long rangeEnd)
        Add to the current bitmap all integers in [rangeStart,rangeEnd).
        Specified by:
        add in interface BitmapDataProvider
        Parameters:
        rangeStart - inclusive beginning of range
        rangeEnd - exclusive ending of range

      • add

        @Deprecated
public void add​(int rangeStart,
                int rangeEnd)
        Deprecated.
        use the version where longs specify the range
        Add to the current bitmap all integers in [rangeStart,rangeEnd).
        Parameters:
        rangeStart - inclusive beginning of range
        rangeEnd - exclusive ending of range

      • and

        public void and​(ImmutableRoaringBitmap array)
        In-place bitwise AND (intersection) operation. The current bitmap is modified.
        Parameters:
        array - other bitmap

      • andNot

        public void andNot​(ImmutableRoaringBitmap x2)
        In-place bitwise ANDNOT (difference) operation. The current bitmap is modified.
        Parameters:
        x2 - other bitmap

      • orNot

        public void orNot​(ImmutableRoaringBitmap other,
                  long rangeEnd)
        In-place bitwise ORNOT operation. The current bitmap is modified.
        Parameters:
        other - the other bitmap
        rangeEnd - end point of the range (exclusive).

      • checkedAdd

        public boolean checkedAdd​(int x)
        Add the value to the container (set the value to "true"), whether it already appears or not.
        Parameters:
        x - integer value
        Returns:
        true if the added int wasn't already contained in the bitmap. False otherwise.

      • checkedRemove

        public boolean checkedRemove​(int x)
        If present remove the specified integer (effectively, sets its bit value to false)
        Parameters:
        x - integer value representing the index in a bitmap
        Returns:
        true if the unset bit was already in the bitmap

      • clear

        public void clear()
        reset to an empty bitmap; result occupies as much space a newly created bitmap.

      • deserialize

        public void deserialize​(java.io.DataInput in)
                 throws java.io.IOException
        Deserialize the bitmap (retrieve from the input stream). The current bitmap is overwritten. See format specification at https://github.com/RoaringBitmap/RoaringFormatSpec
        Parameters:
        in - the DataInput stream
        Throws:
        java.io.IOException - Signals that an I/O exception has occurred.

      • deserialize

        public void deserialize​(java.nio.ByteBuffer buffer)
                 throws java.io.IOException
        Deserialize (retrieve) this bitmap. See format specification at https://github.com/RoaringBitmap/RoaringFormatSpec The current bitmap is overwritten. It is not necessary that limit() on the input ByteBuffer indicates the end of the serialized data. After loading this RoaringBitmap, you can advance to the rest of the data (if there is more) by setting bbf.position(bbf.position() + bitmap.serializedSizeInBytes()); Note that the input ByteBuffer is effectively copied (with the slice operation) so you should expect the provided ByteBuffer to remain unchanged.
        Parameters:
        buffer - the byte buffer (can be mapped, direct, array backed etc.
        Throws:
        java.io.IOException - Signals that an I/O exception has occurred.

      • flip

        public void flip​(int x)
        Add the value if it is not already present, otherwise remove it.
        Parameters:
        x - integer value

      • flip

        public void flip​(long rangeStart,
                 long rangeEnd)
        Modifies the current bitmap by complementing the bits in the given range, from rangeStart (inclusive) rangeEnd (exclusive).
        Parameters:
        rangeStart - inclusive beginning of range
        rangeEnd - exclusive ending of range

      • flip

        @Deprecated
public void flip​(int rangeStart,
                 int rangeEnd)
        Deprecated.
        use the version where longs specify the range
        Modifies the current bitmap by complementing the bits in the given range, from rangeStart (inclusive) rangeEnd (exclusive).
        Parameters:
        rangeStart - inclusive beginning of range
        rangeEnd - exclusive ending of range

      • getMappeableRoaringArray

        public MutableRoaringArray getMappeableRoaringArray()
        Returns:
        a mutable copy of this bitmap

      • iterator

        public java.util.Iterator<java.lang.Integer> iterator()
        iterate over the positions of the true values.
        Specified by:
        iterator in interface java.lang.Iterable<java.lang.Integer>
        Overrides:
        iterator in class ImmutableRoaringBitmap
        Returns:
        the iterator

      • or

        public void or​(ImmutableRoaringBitmap x2)
        In-place bitwise OR (union) operation. The current bitmap is modified.
        Parameters:
        x2 - other bitmap

      • readExternal

        public void readExternal​(java.io.ObjectInput in)
                  throws java.io.IOException,
                         java.lang.ClassNotFoundException
        Specified by:
        readExternal in interface java.io.Externalizable
        Throws:
        java.io.IOException
        java.lang.ClassNotFoundException

      • remove

        public void remove​(int x)
        If present remove the specified integers (effectively, sets its bit value to false)
        Specified by:
        remove in interface BitmapDataProvider
        Parameters:
        x - integer value representing the index in a bitmap

      • remove

        public void remove​(long rangeStart,
                   long rangeEnd)
        Remove from the current bitmap all integers in [rangeStart,rangeEnd).
        Parameters:
        rangeStart - inclusive beginning of range
        rangeEnd - exclusive ending of range

      • remove

        @Deprecated
public void remove​(int rangeStart,
                   int rangeEnd)
        Deprecated.
        use the version where longs specify the range
        Remove from the current bitmap all integers in [rangeStart,rangeEnd).
        Parameters:
        rangeStart - inclusive beginning of range
        rangeEnd - exclusive ending of range

      • removeRunCompression

        public boolean removeRunCompression()
        Remove run-length encoding even when it is more space efficient
        Returns:
        whether a change was applied

      • repairAfterLazy

        protected void repairAfterLazy()

      • runOptimize

        public boolean runOptimize()
        Use a run-length encoding where it is estimated as more space efficient
        Returns:
        whether a change was applied

      • toImmutableRoaringBitmap

        public ImmutableRoaringBitmap toImmutableRoaringBitmap()
        Convenience method, effectively casts the object to an object of class ImmutableRoaringBitmap. This function is equivalent to : 
         
       (ImmutableRoaringBitmap) bitmap
        Some users would prefer to generate a hard copy of the data. The following code illustrates how to proceed, but note that the resulting copy can be expected to perform significantly worse than the original: the toImmutableRoaringBitmap method is almost free, it uses less memory and it produces a much faster bitmap. 
         
      /////////////
      // Code to create a hard copy of MutableRoaringBitmap to an
      // ImmutableRoaringBitmap.
      // Usage of this code is discouraged because it is expensive
      // and it creates a copy that
      // suffers from more performance overhead than the original.
      /////////////
      import org.roaringbitmap.buffer.*;

      //...

      MutableRoaringBitmap rr = ... // some bitmap
      rr.runOptimize(); // can help compression

      // we are going to create an immutable copy of rr
      ByteBuffer outbb = ByteBuffer.allocate(mrb.serializedSizeInBytes());
      mrb.serialize(outbb);
      outbb.flip();
      ImmutableRoaringBitmap irb = new ImmutableRoaringBitmap(outbb);
        Returns:
        a cast of this object

      • trim

        public void trim()
        Recover allocated but unused memory.
        Specified by:
        trim in interface BitmapDataProvider

      • writeExternal

        public void writeExternal​(java.io.ObjectOutput out)
                   throws java.io.IOException
        Specified by:
        writeExternal in interface java.io.Externalizable
        Throws:
        java.io.IOException

      • xor

        public void xor​(ImmutableRoaringBitmap x2)
        In-place bitwise XOR (symmetric difference) operation. The current bitmap is modified.
        Parameters:
        x2 - other bitmap

      • maximumSerializedSize

        public static long maximumSerializedSize​(int cardinality,
                                         int universe_size)
        Assume that one wants to store "cardinality" integers in [0, universe_size), this function returns an upper bound on the serialized size in bytes. This function is identical to RoaringBitmap.maximumSerializedSize.
        Parameters:
        cardinality - maximal cardinality
        universe_size - maximal value
        Returns:
        upper bound on the serialized size in bytes of the bitmap