An i2c_msg is the low level representation of one segment of an I2C transaction. It is visible to drivers in the () procedure, to userspace from i2c-dev, and to I2C adapter drivers through the .() method.
Except when I2C "protocol mangling" is used, all I2C adapters implement the standard rules for I2C transactions. Each transaction begins with a START. That is followed by the slave address, and a bit encoding read versus write. Then follow all the data bytes, possibly including a byte with SMBus PEC. The transfer terminates with a NAK, or when all those bytes have been transferred and ACKed. If this is the last message in a group, it is followed by a STOP. Otherwise it is followed by the next transaction segment, beginning with a (repeated) START.
Alternatively, when the adapter supports I2C_FUNC_PROTOCOL_MANGLING then passing certain may have changed those standard protocol behaviors. Those flags are only for use with broken/nonconforming slaves, and with adapters which are known to support the specific mangling options they need (one or more of IGNORE_NAK, NO_RD_ACK, NOSTART, and REV_DIR_ADDR).