Merge branch 'a-b-error-correction'

README.adoc

@@ -2,10 +2,9 @@
 
 == Description
 
-`audiowmark` is an Open Source solution for audio watermarking. A sound file
-(typically wav) is read by the software, and a 128-bit message is stored in a
-watermark in the output sound file. For human listeners, the files typically
-sound the same.
+`audiowmark` is an Open Source solution for audio watermarking. A sound file is
+read by the software, and a 128-bit message is stored in a watermark in the
+output sound file. For human listeners, the files typically sound the same.
 
 However, the 128-bit message can be retrieved from the output sound file. Our
 tests show, that even if the file is converted to mp3 or ogg (with bitrate 128
@@ -22,17 +21,94 @@ later. The algorithm used here is inspired by
   Martin Steinebach: Digitale Wasserzeichen für Audiodaten.
   Darmstadt University of Technology 2004, ISBN 3-8322-2507-2
 
-== Adding/Retrieving a Watermark
+== Adding a Watermark
 
 To add a watermark to the soundfile in.wav with a 128-bit message (which is
 specified as hex-string):
 
-  audiowmark add in.wav out.wav 0123456789abcdef0011223344556677
+[subs=+quotes]
+....
+  *$ audiowmark add in.wav out.wav 0123456789abcdef0011223344556677*
+  Input:        in.wav
+  Output:       out.wav
+  Message:      0123456789abcdef0011223344556677
+  Strength:     10
 
-To get the 128-bit message from the watermarked file, use:
+  Time:         3:59
+  Sample Rate:  48000
+  Channels:     2
+  Data Blocks:  4
+  Volume Norm:  0.987 (-0.12 dB)
+....
+
+The most important options for adding a watermark are:
+
+--key <filename>::
+Use watermarking key from file <filename> (see <<key>>).
 
-  audiowmark get out.wav
+--strength <s>::
+Set the watermarking strength (see <<strength>>).
+
+== Retrieving a Watermark
+
+To get the 128-bit message from the watermarked file, use:
 
+[subs=+quotes]
+....
+  *$ audiowmark get out.wav*
+  pattern  0:05 0123456789abcdef0011223344556677 1.324 0.059 A
+  pattern  0:57 0123456789abcdef0011223344556677 1.413 0.112 B
+  pattern  0:57 0123456789abcdef0011223344556677 1.368 0.086 AB
+  pattern  1:49 0123456789abcdef0011223344556677 1.302 0.098 A
+  pattern  2:40 0123456789abcdef0011223344556677 1.361 0.093 B
+  pattern  2:40 0123456789abcdef0011223344556677 1.331 0.096 AB
+  pattern   all 0123456789abcdef0011223344556677 1.350 0.054
+....
+The output of `audiowmark get` is designed to be machine readable. Each line
+that starts with `pattern` contains one decoded message. The fields are
+seperated by one or more space characters. The first field is a *timestamp*
+indicating the position of the data block. The second field is the *decoded
+message*. For most purposes this is all you need to know.
+
+The software was designed under the assumption that you - the user - will be
+able to decide whether a message is correct or not. To do this, on watermarking
+song files, you could list each message you embedded in a database. During
+retrieval, you should look up each pattern `audiowmark get` outputs in the
+database. If the message is not found, then you should assume that a decoding
+error occurred. In our example each pattern was decoded correctly, because
+the watermark was not damaged at all, but if you for instance use lossy
+compression (with a low bitrate), it may happen that only some of the decoded
+patterns are correct. Or none, if the watermark was damaged too much.
+
+The third field is the *sync score* (higher is better). The synchronization
+algorithm tries to find valid data blocks in the audio file, that become
+candidates for decoding.
+
+The fourth field is the *decoding error* (lower is better). During message
+decoding, we use convolutional codes for error correction, to make the
+watermarking more robust.
+
+The fifth field is the *block type*. There are two types of data blocks,
+A blocks and B blocks. A single data block can be decoded alone, as it
+contains a complete message. However, if during watermark detection an
+A block followed by a B block was found, these two can be decoded
+together (then this field will be AB), resulting in even higher error
+correction capacity than one block alone would have.
+
+To improve the error correction capacity even further, the `all` pattern
+combines all data blocks that are available. The combined decoded
+message will often be the most reliable result (meaning that even if all
+other patterns were incorrect, this could still be right).
+
+The most important options for getting a watermark are:
+
+--key <filename>::
+Use watermarking key from file <filename> (see <<key>>).
+
+--strength <s>::
+Set the watermarking strength (see <<strength>>).
+
+[[key]]
 == Watermark Key
 
 Since the software is Open Source, a watermarking key should be used to ensure
@@ -52,6 +128,7 @@ and can be used for the add/get commands as follows:
   audiowmark add --key test.key in.wav out.wav 0123456789abcdef0011223344556677
   audiowmark get --key test.key out.wav
 
+[[strength]]
 == Watermark Strength
 
 The watermark strength parameter affects how much the watermarking algorithm
@@ -74,7 +151,7 @@ watermark. Fractional strengths (like 7.5) are possible.
 
 == Dependencies
 
-If you compile from source, audiowmark needs the follwing libraries:
+If you compile from source, audiowmark needs the following libraries:
 
 * libfftw3
 * libsndfile

src/audiowmark.cc

@@ -42,13 +42,14 @@ namespace Params
   static int sync_frames_per_bit = 85;
   static int sync_search_step    = 256;
   static int sync_search_fine    = 8;
-  static double sync_threshold1  = 0.5; // minimum grid quality value (search_step grid)
   static double sync_threshold2  = 0.7; // minimum refined quality
 
   static size_t frames_pad_start = 250; // padding at start, in case track starts with silence
   static int    mark_sample_rate = 44100; // watermark generation and detection sample rate
 
   static int test_cut            = 0; // for sync test
+  static bool test_no_sync       = false; // disable sync
+  static int test_truncate       = 0;
 }
 
 void
@@ -60,25 +61,17 @@ print_usage()
   printf ("  * create a watermarked wav file with a message\n");
   printf ("    audiowmark add <input_wav> <watermarked_wav> <message_hex>\n");
   printf ("\n");
-  printf ("  * blind decoding (retrieve message without original file)\n");
+  printf ("  * retrieve message\n");
   printf ("    audiowmark get <watermarked_wav>\n");
   printf ("\n");
-  printf ("  * compute bit error rate for blind decoding\n");
+  printf ("  * compare watermark message with expected message\n");
   printf ("    audiowmark cmp <watermarked_wav> <message_hex>\n");
   printf ("\n");
-#if 0
-  printf ("  * retrieve message with original file\n");
-  printf ("    audiowmark get-delta <input_wav> <watermarked_wav>\n");
-  printf ("\n");
-  printf ("  * compute bit error rate for decoding with original file\n");
-  printf ("    audiowmark cmp-delta <input_wav> <watermarked_wav> <message_hex>\n");
-  printf ("\n");
-#endif
   printf ("  * generate 128-bit watermarking key, to be used with --key option\n");
   printf ("    audiowmark gen-key <key_file>\n");
   printf ("\n");
   printf ("Global options:\n");
-  printf ("  --strength            set watermark strength              [%.6g]\n", Params::water_delta * 1000);
+  printf ("  --strength <s>        set watermark strength              [%.6g]\n", Params::water_delta * 1000);
   printf ("  --linear              disable non-linear bit storage\n");
   printf ("  --key <file>          load watermarking key from file\n");
 }
@@ -187,6 +180,14 @@ parse_options (int   *argc_p,
 	{
           Params::test_cut = atoi (opt_arg);
 	}
+      else if (check_arg (argc, argv, &i, "--test-no-sync"))
+        {
+          Params::test_no_sync = true;
+        }
+      else if (check_arg (argc, argv, &i, "--test-truncate", &opt_arg))
+	{
+          Params::test_truncate = atoi (opt_arg);
+	}
     }
 
   /* resort argc/argv */
@@ -279,7 +280,7 @@ randomize_bit_order (const vector<T>& bit_vec, bool encode)
 }
 
 vector<vector<complex<float>>>
-compute_frame_ffts (const WavData& wav_data, size_t start_index, size_t frame_count)
+compute_frame_ffts (const WavData& wav_data, size_t start_index, size_t frame_count, const vector<int>& want_frames)
 {
   vector<vector<complex<float>>> fft_out;
 
@@ -312,26 +313,35 @@ compute_frame_ffts (const WavData& wav_data, size_t start_index, size_t frame_co
 
   for (size_t f = 0; f < frame_count; f++)
     {
-      for (int ch = 0; ch < wav_data.n_channels(); ch++)
+      if (!want_frames.empty() && !want_frames[f])
+        {
+          /* skip fft calculation completely if frame is not in want_frames */
+          for (int ch = 0; ch < wav_data.n_channels(); ch++)
+            fft_out.emplace_back();
+        }
+      else
         {
-          const auto& samples = wav_data.samples();
+          for (int ch = 0; ch < wav_data.n_channels(); ch++)
+            {
+              const auto& samples = wav_data.samples();
 
-          size_t pos = (start_index + f * Params::frame_size) * wav_data.n_channels() + ch;
-          assert (pos + (Params::frame_size - 1) * wav_data.n_channels() < samples.size());
+              size_t pos = (start_index + f * Params::frame_size) * wav_data.n_channels() + ch;
+              assert (pos + (Params::frame_size - 1) * wav_data.n_channels() < samples.size());
 
-          /* deinterleave frame data and apply window */
-          for (size_t x = 0; x < Params::frame_size; x++)
-            {
-              frame[x] = samples[pos] * window[x];
-              pos += wav_data.n_channels();
-            }
-          /* FFT transform */
-          fftar_float (Params::frame_size, frame, frame_fft);
+              /* deinterleave frame data and apply window */
+              for (size_t x = 0; x < Params::frame_size; x++)
+                {
+                  frame[x] = samples[pos] * window[x];
+                  pos += wav_data.n_channels();
+                }
+              /* FFT transform */
+              fftar_float (Params::frame_size, frame, frame_fft);
 
-          /* complex<float> and frame_fft have the same layout in memory */
-          const complex<float> *first = (complex<float> *) frame_fft;
-          const complex<float> *last  = first + Params::frame_size / 2 + 1;
-          fft_out.emplace_back (first, last);
+              /* complex<float> and frame_fft have the same layout in memory */
+              const complex<float> *first = (complex<float> *) frame_fft;
+              const complex<float> *last  = first + Params::frame_size / 2 + 1;
+              fft_out.emplace_back (first, last);
+            }
         }
     }
   free_array_float (frame);
@@ -339,6 +349,49 @@ compute_frame_ffts (const WavData& wav_data, size_t start_index, size_t frame_co
   return fft_out;
 }
 
+size_t mark_data_frame_count();
+size_t mark_sync_frame_count();
+
+int
+frame_pos (int f, bool sync)
+{
+  static vector<int> pos_vec;
+
+  if (pos_vec.empty())
+    {
+      int frame_count = mark_data_frame_count() + mark_sync_frame_count();
+      for (int i = 0; i < frame_count; i++)
+        pos_vec.push_back (i);
+
+      Random random (0, Random::Stream::frame_position);
+      random.shuffle (pos_vec);
+    }
+  if (sync)
+    {
+      assert (f >= 0 && size_t (f) < mark_sync_frame_count());
+
+      return pos_vec[f];
+    }
+  else
+    {
+      assert (f >= 0 && size_t (f) < mark_data_frame_count());
+
+      return pos_vec[f + mark_sync_frame_count()];
+    }
+}
+
+int
+sync_frame_pos (int f)
+{
+  return frame_pos (f, true);
+}
+
+int
+data_frame_pos (int f)
+{
+  return frame_pos (f, false);
+}
+
 void
 mark_bit_linear (int f, const vector<complex<float>>& fft_out, vector<complex<float>>& fft_delta_spect, int data_bit, Random::Stream random_stream)
 {
@@ -382,7 +435,7 @@ mark_bit_linear (int f, const vector<complex<float>>& fft_out, vector<complex<fl
 size_t
 mark_data_frame_count()
 {
-  return conv_code_size (Params::payload_size) * Params::frames_per_bit;
+  return conv_code_size (ConvBlockType::a, Params::payload_size) * Params::frames_per_bit;
 }
 
 struct MixEntry
@@ -405,7 +458,7 @@ gen_mix_entries()
 
       assert (up.size() == down.size());
       for (size_t i = 0; i < up.size(); i++)
-        mix_entries.push_back ({ f, up[i], down[i] });
+        mix_entries.push_back ({ data_frame_pos (f), up[i], down[i] });
     }
   Random random (/* seed */ 0, Random::Stream::mix);
   random.shuffle (mix_entries);
@@ -469,7 +522,7 @@ mark_data (const WavData& wav_data, int start_frame, const vector<vector<complex
         {
           for (int ch = 0; ch < wav_data.n_channels(); ch++)
             {
-              size_t index = (start_frame + f) * wav_data.n_channels() + ch;
+              size_t index = (start_frame + data_frame_pos (f)) * wav_data.n_channels() + ch;
 
               mark_bit_linear (f, fft_out[index], fft_delta_spect[index], bitvec[f / Params::frames_per_bit], Random::Stream::data_up_down);
             }
@@ -484,7 +537,7 @@ mark_sync_frame_count()
 }
 
 void
-mark_sync (const WavData& wav_data, int start_frame, const vector<vector<complex<float>>>& fft_out, vector<vector<complex<float>>>& fft_delta_spect)
+mark_sync (const WavData& wav_data, int start_frame, const vector<vector<complex<float>>>& fft_out, vector<vector<complex<float>>>& fft_delta_spect, int ab)
 {
   assert (fft_out.size() >= (start_frame + mark_sync_frame_count()) * wav_data.n_channels());
 
@@ -495,8 +548,8 @@ mark_sync (const WavData& wav_data, int start_frame, const vector<vector<complex
     {
       for (int ch = 0; ch < wav_data.n_channels(); ch++)
         {
-          size_t index = (start_frame + f) * wav_data.n_channels() + ch;
-          int    data_bit = (f / Params::sync_frames_per_bit) & 1; /* write 010101 */
+          size_t index = (start_frame + sync_frame_pos (f)) * wav_data.n_channels() + ch;
+          int    data_bit = (f / Params::sync_frames_per_bit + ab) & 1; /* write 010101 for a block, 101010 for b block */
 
           mark_bit_linear (f, fft_out[index], fft_delta_spect[index], data_bit, Random::Stream::sync_up_down);
         }
@@ -608,10 +661,8 @@ add_watermark (const string& infile, const string& outfile, const string& bits)
   printf ("Strength:     %.6g\n\n", Params::water_delta * 1000);
 
   /* add forward error correction, bitvec will now be a lot larger */
-  bitvec = randomize_bit_order (conv_encode (bitvec), /* encode */ true);
-
-  /* pad with zeros to match block_size */
-  bitvec.resize (mark_data_frame_count() / Params::frames_per_bit);
+  auto bitvec_a = randomize_bit_order (conv_encode (ConvBlockType::a, bitvec), /* encode */ true);
+  auto bitvec_b = randomize_bit_order (conv_encode (ConvBlockType::b, bitvec), /* encode */ true);
 
   WavData orig_wav_data;
   if (!orig_wav_data.load (infile))
@@ -647,7 +698,7 @@ add_watermark (const string& infile, const string& outfile, const string& bits)
   /* we have extra space for the padded wave data -> truncated before save */
   vector<float> out_signal (wav_data.n_values());
 
-  vector<vector<complex<float>>> fft_out = compute_frame_ffts (wav_data, 0, frame_count (wav_data));
+  vector<vector<complex<float>>> fft_out = compute_frame_ffts (wav_data, 0, frame_count (wav_data), /* want all frames */ {});
   vector<vector<complex<float>>> fft_delta_spect;
   for (int f = 0; f < frame_count (wav_data); f++)
     {
@@ -667,13 +718,12 @@ add_watermark (const string& infile, const string& outfile, const string& bits)
   /* embed sync|data|sync|data|... */
   while (frame_index + (mark_sync_frame_count() + mark_data_frame_count()) < size_t (frame_count (wav_data)))
     {
-      mark_sync (wav_data, frame_index, fft_out, fft_delta_spect);
-      frame_index += mark_sync_frame_count();
+      mark_sync (wav_data, frame_index, fft_out, fft_delta_spect, (data_blocks & 1));
+      mark_data (wav_data, frame_index, fft_out, fft_delta_spect, (data_blocks & 1) ? bitvec_b : bitvec_a);
 
-      data_blocks++;
+      frame_index += mark_sync_frame_count() + mark_data_frame_count();
 
-      mark_data (wav_data, frame_index, fft_out, fft_delta_spect, bitvec);
-      frame_index += mark_data_frame_count();
+      data_blocks++;
     }
   /* padding at end */
   while (frame_index < size_t (frame_count (wav_data)))
@@ -823,11 +873,11 @@ normalize_soft_bits (const vector<float>& soft_bits)
 }
 
 vector<float>
-mix_decode (vector<vector<complex<float>>>& fft_out, vector<vector<complex<float>>>& fft_orig_out, int n_channels)
+mix_decode (vector<vector<complex<float>>>& fft_out, int n_channels)
 {
   vector<float> raw_bit_vec;
 
-  const int frame_count = fft_out.size() / n_channels;
+  const int frame_count = mark_data_frame_count();
 
   vector<MixEntry> mix_entries = gen_mix_entries();
 
@@ -847,12 +897,6 @@ mix_decode (vector<vector<complex<float>>>& fft_out, vector<vector<complex<float
 
               umag += db_from_factor (abs (fft_out[index][u]), min_db);
               dmag += db_from_factor (abs (fft_out[index][d]), min_db);
-
-              if (index < fft_orig_out.size()) /* non-blind decode? */
-                {
-                  umag -= db_from_factor (abs (fft_orig_out[index][u]), min_db);
-                  dmag -= db_from_factor (abs (fft_orig_out[index][d]), min_db);
-                }
             }
         }
       if ((f % Params::frames_per_bit) == (Params::frames_per_bit - 1))
@@ -866,36 +910,28 @@ mix_decode (vector<vector<complex<float>>>& fft_out, vector<vector<complex<float
 }
 
 vector<float>
-linear_decode (vector<vector<complex<float>>>& fft_out, vector<vector<complex<float>>>& fft_orig_out, int n_channels)
+linear_decode (vector<vector<complex<float>>>& fft_out, int n_channels)
 {
   vector<float> raw_bit_vec;
 
+  const int frame_count = mark_data_frame_count();
+
   double umag = 0, dmag = 0;
-  const int frame_count = fft_out.size() / n_channels;
   for (int f = 0; f < frame_count; f++)
     {
       for (int ch = 0; ch < n_channels; ch++)
         {
-          const size_t index = f * n_channels + ch;
+          const size_t index = data_frame_pos (f) * n_channels + ch;
           vector<int> up;
           vector<int> down;
           get_up_down (f, up, down, Random::Stream::data_up_down);
 
           const double min_db = -96;
           for (auto u : up)
-            {
-              umag += db_from_factor (abs (fft_out[index][u]), min_db);
+            umag += db_from_factor (abs (fft_out[index][u]), min_db);
 
-              if (index < fft_orig_out.size())
-                umag -= db_from_factor (abs (fft_orig_out[index][u]), min_db);
-            }
           for (auto d : down)
-            {
-              dmag += db_from_factor (abs (fft_out[index][d]), min_db);
-
-              if (index < fft_orig_out.size())
-                dmag -= db_from_factor (abs (fft_orig_out[index][d]), min_db);
-            }
+            dmag += db_from_factor (abs (fft_out[index][d]), min_db);
         }
       if ((f % Params::frames_per_bit) == (Params::frames_per_bit - 1))
         {
@@ -943,17 +979,17 @@ class SyncFinder
             get_up_down (f + bit * Params::sync_frames_per_bit, frame_up, frame_down, Random::Stream::sync_up_down);
 
             for (auto u : frame_up)
-              up[bit].push_back (u - Params::min_band + f * n_bands * wav_data.n_channels());
+              up[bit].push_back (u - Params::min_band + sync_frame_pos (f + bit * Params::sync_frames_per_bit) * n_bands * wav_data.n_channels());
 
             for (auto d : frame_down)
-              down[bit].push_back (d - Params::min_band + f * n_bands * wav_data.n_channels());
+              down[bit].push_back (d - Params::min_band + sync_frame_pos (f + bit * Params::sync_frames_per_bit) * n_bands * wav_data.n_channels());
           }
         sort (up[bit].begin(), up[bit].end());
         sort (down[bit].begin(), down[bit].end());
       }
   }
   double
-  sync_decode (const WavData& wav_data, const size_t start_frame, const vector<float>& fft_out_db)
+  sync_decode (const WavData& wav_data, const size_t start_frame, const vector<float>& fft_out_db, ConvBlockType *block_type)
   {
     double sync_quality = 0;
 
@@ -964,7 +1000,7 @@ class SyncFinder
 
         for (int ch = 0; ch < wav_data.n_channels(); ch++)
           {
-            const int index = (((bit * Params::sync_frames_per_bit) + start_frame) * wav_data.n_channels() + ch) * n_bands;
+            const int index = (start_frame * wav_data.n_channels() + ch) * n_bands;
 
             for (size_t i = 0; i < up[bit].size(); i++)
               {
@@ -989,12 +1025,23 @@ class SyncFinder
       }
     sync_quality /= Params::sync_bits;
     sync_quality = normalize_sync_quality (sync_quality);
-    return sync_quality;
+
+    if (sync_quality < 0)
+      {
+        *block_type = ConvBlockType::b;
+        return -sync_quality;
+      }
+    else
+      {
+        *block_type = ConvBlockType::a;
+        return sync_quality;
+      }
   }
 public:
   struct Score {
-    size_t index;
-    double quality;
+    size_t        index;
+    double        quality;
+    ConvBlockType block_type;
   };
   vector<Score>
   search (const WavData& wav_data)
@@ -1002,6 +1049,18 @@ public:
     vector<Score> result_scores;
     vector<Score> sync_scores;
 
+    if (Params::test_no_sync)
+      {
+        const size_t expect0 = Params::frames_pad_start * Params::frame_size;
+        const size_t expect_step = (mark_sync_frame_count() + mark_data_frame_count()) * Params::frame_size;
+        const size_t expect_end = frame_count (wav_data) * Params::frame_size;
+
+        int ab = 0;
+        for (size_t expect_index = expect0; expect_index + expect_step < expect_end; expect_index += expect_step)
+          result_scores.push_back (Score { expect_index, 1.0, (ab++ & 1) ? ConvBlockType::b : ConvBlockType::a });
+
+        return result_scores;
+      }
     init_up_down (wav_data);
 
     vector<float> fft_db;
@@ -1010,23 +1069,38 @@ public:
     size_t n_bands = Params::max_band - Params::min_band + 1;
     for (size_t sync_shift = 0; sync_shift < Params::frame_size; sync_shift += Params::sync_search_step)
       {
-        sync_fft (wav_data, sync_shift, frame_count (wav_data) - 1, fft_db);
+        sync_fft (wav_data, sync_shift, frame_count (wav_data) - 1, fft_db, /* want all frames */ {});
         for (int start_frame = 0; start_frame < frame_count (wav_data); start_frame++)
           {
             const size_t sync_index = start_frame * Params::frame_size + sync_shift;
-            if ((start_frame + mark_sync_frame_count()) * wav_data.n_channels() * n_bands < fft_db.size())
+            if ((start_frame + mark_sync_frame_count() + mark_data_frame_count()) * wav_data.n_channels() * n_bands < fft_db.size())
               {
-                double quality = sync_decode (wav_data, start_frame, fft_db);
+                ConvBlockType block_type;
+                double quality = sync_decode (wav_data, start_frame, fft_db, &block_type);
                 // printf ("%zd %f\n", sync_index, quality);
-                sync_scores.emplace_back (Score { sync_index, quality });
+                sync_scores.emplace_back (Score { sync_index, quality, block_type });
               }
           }
       }
     sort (sync_scores.begin(), sync_scores.end(), [] (const Score& a, const Score &b) { return a.index < b.index; });
+
+    vector<int> want_frames (mark_sync_frame_count() + mark_data_frame_count());
+    for (size_t f = 0; f < mark_sync_frame_count(); f++)
+      want_frames[sync_frame_pos (f)] = 1;
+
+    /* for strength 8 and above:
+     *   -> more false positive candidates are rejected, so we can use a lower threshold
+     *
+     * for strength 7 and below:
+     *   -> we need a higher threshold, because otherwise watermark detection takes too long
+     */
+    const double strength = Params::water_delta * 1000;
+    const double sync_threshold1 = strength > 7.5 ? 0.4 : 0.5;
+
     for (size_t i = 0; i < sync_scores.size(); i++)
       {
         // printf ("%zd %f\n", sync_scores[i].index, sync_scores[i].quality);
-        if (sync_scores[i].quality > Params::sync_threshold1)
+        if (sync_scores[i].quality > sync_threshold1)
           {
             double q_last = -1;
             double q_next = -1;
@@ -1042,17 +1116,19 @@ public:
                 //printf ("%zd %s %f", sync_scores[i].index, find_closest_sync (sync_scores[i].index), sync_scores[i].quality);
 
                 // refine match
-                double best_quality = sync_scores[i].quality;
-                size_t best_index   = sync_scores[i].index;
+                double best_quality       = sync_scores[i].quality;
+                size_t best_index         = sync_scores[i].index;
+                ConvBlockType best_block_type = sync_scores[i].block_type; /* doesn't really change during refinement */
 
                 int start = std::max (int (sync_scores[i].index) - Params::sync_search_step, 0);
                 int end   = sync_scores[i].index + Params::sync_search_step;
                 for (int fine_index = start; fine_index <= end; fine_index += Params::sync_search_fine)
                   {
-                    sync_fft (wav_data, fine_index, mark_sync_frame_count(), fft_db);
+                    sync_fft (wav_data, fine_index, mark_sync_frame_count() + mark_data_frame_count(), fft_db, want_frames);
                     if (fft_db.size())
                       {
-                        double q = sync_decode (wav_data, 0, fft_db);
+                        ConvBlockType block_type;
+                        double        q = sync_decode (wav_data, 0, fft_db, &block_type);
 
                         if (q > best_quality)
                           {
@@ -1063,7 +1139,7 @@ public:
                   }
                 //printf (" => refined: %zd %s %f\n", best_index, find_closest_sync (best_index), best_quality);
                 if (best_quality > Params::sync_threshold2)
-                  result_scores.push_back (Score { best_index, best_quality });
+                  result_scores.push_back (Score { best_index, best_quality, best_block_type });
               }
           }
       }
@@ -1071,17 +1147,26 @@ public:
   }
 private:
   void
-  sync_fft (const WavData& wav_data, size_t index, size_t count, vector<float>& fft_out_db)
+  sync_fft (const WavData& wav_data, size_t index, size_t count, vector<float>& fft_out_db, const vector<int>& want_frames)
   {
     fft_out_db.clear();
 
     /* computing db-magnitude is expensive, so we better do it here */
-    for (const vector<complex<float>>& spect : compute_frame_ffts (wav_data, index, count))
+    vector<vector<complex<float>>> fft_out = compute_frame_ffts (wav_data, index, count, want_frames);
+    for (size_t p = 0; p < fft_out.size(); p++)
       {
         const double min_db = -96;
 
-        for (int i = Params::min_band; i <= Params::max_band; i++)
-          fft_out_db.push_back (db_from_factor (abs (spect[i]), min_db));
+        if (!fft_out[p].size()) // not in want_frames?
+          {
+            for (int i = Params::min_band; i <= Params::max_band; i++)
+              fft_out_db.push_back (min_db);
+          }
+        else
+          {
+            for (int i = Params::min_band; i <= Params::max_band; i++)
+              fft_out_db.push_back (db_from_factor (abs (fft_out[p][i]), min_db));
+          }
       }
   }
 
@@ -1114,25 +1199,29 @@ decode_and_report (const WavData& wav_data, const string& orig_pattern)
   SyncFinder                sync_finder;
   vector<SyncFinder::Score> sync_scores = sync_finder.search (wav_data);
 
-  auto decode_single = [&] (const vector<float>& raw_bit_vec, SyncFinder::Score sync_score)
+  auto report_pattern = [&] (SyncFinder::Score sync_score, const vector<int>& bit_vec, float decode_error)
   {
-    assert (raw_bit_vec.size() == conv_code_size (Params::payload_size));
-
-    vector<float> soft_bit_vec = normalize_soft_bits (raw_bit_vec);
-
-    float decode_error = 0;
-    vector<int> bit_vec = conv_decode_soft (randomize_bit_order (soft_bit_vec, /* encode */ false), &decode_error);
-
     if (sync_score.index)
       {
+        const char *block_str = nullptr;
+
+        switch (sync_score.block_type)
+          {
+            case ConvBlockType::a:  block_str = "A";
+                                    break;
+            case ConvBlockType::b:  block_str = "B";
+                                    break;
+            case ConvBlockType::ab: block_str = "AB";
+                                    break;
+          }
         const int seconds = sync_score.index / wav_data.sample_rate();
-        printf ("pattern %2d:%02d %s %.3f %.3f\n", seconds / 60, seconds % 60, bit_vec_to_str (bit_vec).c_str(), sync_score.quality, decode_error);
+        printf ("pattern %2d:%02d %s %.3f %.3f %s\n", seconds / 60, seconds % 60, bit_vec_to_str (bit_vec).c_str(),
+                sync_score.quality, decode_error, block_str);
       }
     else /* this is the combined pattern "all" */
       {
         printf ("pattern   all %s %.3f %.3f\n", bit_vec_to_str (bit_vec).c_str(), sync_score.quality, decode_error);
       }
-
     if (!orig_pattern.empty())
       {
         bool        match = true;
@@ -1148,39 +1237,88 @@ decode_and_report (const WavData& wav_data, const string& orig_pattern)
     total_count++;
   };
 
-  vector<float> raw_bit_vec_all (conv_code_size (Params::payload_size));
+  vector<float> raw_bit_vec_all (conv_code_size (ConvBlockType::ab, Params::payload_size));
+  vector<int>   raw_bit_vec_norm (2);
+
   SyncFinder::Score score_all { 0, 0 };
+  SyncFinder::Score score_ab  { 0, 0, ConvBlockType::ab };
 
+  ConvBlockType last_block_type = ConvBlockType::b;
+  vector<vector<float>> ab_raw_bit_vec (2);
+  vector<float>         ab_quality (2);
   for (auto sync_score : sync_scores)
     {
-      const size_t count = mark_data_frame_count();
-      const size_t index = sync_score.index + (mark_sync_frame_count() * Params::frame_size);
+      const size_t count = mark_sync_frame_count() + mark_data_frame_count();
+      const size_t index = sync_score.index;
+      const int    ab = (sync_score.block_type == ConvBlockType::b); /* A -> 0, B -> 1 */
 
-      auto fft_range_out = compute_frame_ffts (wav_data, index, count);
+      auto fft_range_out = compute_frame_ffts (wav_data, index, count, /* want all frames */ {});
       if (fft_range_out.size())
         {
-          vector<vector<complex<float>>> junk;
-
+          /* ---- retrieve bits from watermark ---- */
           vector<float> raw_bit_vec;
           if (Params::mix)
             {
-              raw_bit_vec = mix_decode (fft_range_out, junk, wav_data.n_channels());
+              raw_bit_vec = mix_decode (fft_range_out, wav_data.n_channels());
             }
           else
             {
-              raw_bit_vec = linear_decode (fft_range_out, junk, wav_data.n_channels());
+              raw_bit_vec = linear_decode (fft_range_out, wav_data.n_channels());
             }
-          decode_single (raw_bit_vec, sync_score);
+          assert (raw_bit_vec.size() == conv_code_size (ConvBlockType::a, Params::payload_size));
 
+          raw_bit_vec = randomize_bit_order (raw_bit_vec, /* encode */ false);
+
+          /* ---- deal with this pattern ---- */
+          float decode_error = 0;
+          vector<int> bit_vec = conv_decode_soft (sync_score.block_type, normalize_soft_bits (raw_bit_vec), &decode_error);
+
+          report_pattern (sync_score, bit_vec, decode_error);
+
+          /* ---- update "all" pattern ---- */
           score_all.quality += sync_score.quality;
-          for (size_t i = 0; i < raw_bit_vec_all.size(); i++)
-            raw_bit_vec_all[i] += raw_bit_vec[i];
+
+          for (size_t i = 0; i < raw_bit_vec.size(); i++)
+            {
+              raw_bit_vec_all[i * 2 + ab] += raw_bit_vec[i];
+            }
+          raw_bit_vec_norm[ab]++;
+
+          /* ---- if last block was A & this block is B => deal with combined AB block */
+          ab_raw_bit_vec[ab] = raw_bit_vec;
+          ab_quality[ab]     = sync_score.quality;
+          if (last_block_type == ConvBlockType::a && sync_score.block_type == ConvBlockType::b)
+            {
+              /* join A and B block -> AB block */
+              vector<float> ab_bits (raw_bit_vec.size() * 2);
+              for (size_t i = 0; i <  raw_bit_vec.size(); i++)
+                {
+                  ab_bits[i * 2] = ab_raw_bit_vec[0][i];
+                  ab_bits[i * 2 + 1] = ab_raw_bit_vec[1][i];
+                }
+              vector<int> bit_vec = conv_decode_soft (ConvBlockType::ab, normalize_soft_bits (ab_bits), &decode_error);
+              score_ab.index = sync_score.index;
+              score_ab.quality = (ab_quality[0] + ab_quality[1]) / 2;
+              report_pattern (score_ab, bit_vec, decode_error);
+            }
+          last_block_type = sync_score.block_type;
         }
     }
   if (total_count > 1) /* all pattern: average soft bits of all watermarks and decode */
     {
-      score_all.quality /= total_count;
-      decode_single (raw_bit_vec_all, score_all);
+      for (size_t i = 0; i < raw_bit_vec_all.size(); i += 2)
+        {
+          raw_bit_vec_all[i]     /= max (raw_bit_vec_norm[0], 1); /* normalize A soft bits with number of A blocks */
+          raw_bit_vec_all[i + 1] /= max (raw_bit_vec_norm[1], 1); /* normalize B soft bits with number of B blocks */
+        }
+      score_all.quality /= raw_bit_vec_norm[0] + raw_bit_vec_norm[1];
+
+      vector<float> soft_bit_vec = normalize_soft_bits (raw_bit_vec_all);
+
+      float decode_error = 0;
+      vector<int> bit_vec = conv_decode_soft (ConvBlockType::ab, soft_bit_vec, &decode_error);
+
+      report_pattern (score_all, bit_vec, decode_error);
     }
 
   if (!orig_pattern.empty())
@@ -1218,6 +1356,17 @@ get_watermark (const string& infile, const string& orig_pattern)
       return 1;
     }
 
+  if (Params::test_truncate)
+    {
+      const size_t  want_n_samples = wav_data.sample_rate() * wav_data.n_channels() * Params::test_truncate;
+      vector<float> short_samples  = wav_data.samples();
+
+      if (want_n_samples < short_samples.size())
+        {
+          short_samples.resize (want_n_samples);
+          wav_data.set_samples (short_samples);
+        }
+    }
   if (wav_data.sample_rate() == Params::mark_sample_rate)
     {
       return decode_and_report (wav_data, orig_pattern);
@@ -1228,34 +1377,6 @@ get_watermark (const string& infile, const string& orig_pattern)
     }
 }
 
-int
-get_watermark_delta (const string& origfile, const string& infile, const string& orig_pattern)
-{
-  WavData orig_wav_data;
-  if (!orig_wav_data.load (origfile))
-    {
-      fprintf (stderr, "audiowmark: error loading %s: %s\n", origfile.c_str(), orig_wav_data.error_blurb());
-      return 1;
-    }
-
-  WavData wav_data;
-  if (!wav_data.load (infile))
-    {
-      fprintf (stderr, "audiowmark: error loading %s: %s\n", infile.c_str(), wav_data.error_blurb());
-      return 1;
-    }
-
-  /*
-  vector<vector<complex<float>>> fft_out = compute_frame_ffts (wav_data);
-  vector<vector<complex<float>>> fft_orig_out = compute_frame_ffts (orig_wav_data);
-
-  return decode_and_report (wav_data, orig_pattern, fft_out, fft_orig_out);
-  */
-  /* FIXME? */
-  printf ("delta decoding currently not supported\n");
-  return 1;
-}
-
 int
 gentest (const string& infile, const string& outfile)
 {
@@ -1364,14 +1485,6 @@ main (int argc, char **argv)
     {
       cut_start (argv[2], argv[3], argv[4]);
     }
-  else if (op == "get-delta" && argc == 4)
-    {
-      return get_watermark_delta (argv[2], argv[3], /* no ber */ "");
-    }
-  else if (op == "cmp-delta" && argc == 5)
-    {
-      return get_watermark_delta (argv[2], argv[3], argv[4]);
-    }
   else if (op == "gen-key" && argc == 3)
     {
       return gen_key (argv[2]);

src/ber-test.sh

 #!/bin/bash
 
 TRANSFORM=$1
+if [ "x$AWM_TRUNCATE" != "x" ]; then
+  AWM_REPORT=truncv
+fi
 if [ "x$AWM_SET" == "x" ]; then
   AWM_SET=small
 fi
@@ -29,7 +32,7 @@ fi
 do
   for SEED in $AWM_SEEDS
   do
-    echo $i
+    echo in_file $i
 
     if [ "x$AWM_RAND_PATTERN" != "x" ]; then
       # random pattern, 128 bit
@@ -43,12 +46,14 @@ do
       # pseudo random pattern, 128 bit
       PATTERN=4e1243bd22c66e76c2ba9eddc1f91394
     fi
-
+    echo in_pattern $PATTERN
+    echo in_flags $AWM_PARAMS --test-key $SEED
     audiowmark add "$i" ${AWM_FILE}.wav $PATTERN $AWM_PARAMS --test-key $SEED >/dev/null
     if [ "x$AWM_RAND_CUT" != x ]; then
       CUT=$RANDOM
       audiowmark cut-start "${AWM_FILE}.wav" "${AWM_FILE}.wav" $CUT
       TEST_CUT_ARGS="--test-cut $CUT"
+      echo in_cut $CUT
     else
       TEST_CUT_ARGS=""
     fi
@@ -85,18 +90,47 @@ do
       echo "unknown transform $TRANSFORM" >&2
       exit 1
     fi
-    # blind decoding
-    audiowmark cmp $OUT_FILE $PATTERN $AWM_PARAMS --test-key $SEED $TEST_CUT_ARGS
-    # decoding with original
-    # audiowmark cmp-delta "$i" t.wav $PATTERN $AWM_PARAMS --test-key $SEED
+    echo
+    if [ "x$AWM_REPORT" == "xtruncv" ]; then
+      for TRUNC in $AWM_TRUNCATE
+      do
+        audiowmark cmp $OUT_FILE $PATTERN $AWM_PARAMS --test-key $SEED $TEST_CUT_ARGS --test-truncate $TRUNC | sed "s/^/$TRUNC /g"
+        echo
+      done
+    else
+      audiowmark cmp $OUT_FILE $PATTERN $AWM_PARAMS --test-key $SEED $TEST_CUT_ARGS
+      echo
+    fi
+    rm -f ${AWM_FILE}.wav $OUT_FILE # cleanup temp files
   done
 done | {
   if [ "x$AWM_REPORT" == "xfer" ]; then
     awk 'BEGIN { bad = n = 0 } $1 == "match_count" { if ($2 == 0) bad++; n++; } END { print bad, n, bad * 100.0 / n; }'
+  elif [ "x$AWM_REPORT" == "xferv" ]; then
+    awk 'BEGIN { bad = n = 0 } { print "###", $0; } $1 == "match_count" { if ($2 == 0) bad++; n++; } END { print bad, n, bad * 100.0 / n; }'
   elif [ "x$AWM_REPORT" == "xsync" ]; then
     awk 'BEGIN { bad = n = 0 } $1 == "sync_match" { bad += (3 - $2) / 3.0; n++; } END { print bad, n, bad * 100.0 / n; }'
   elif [ "x$AWM_REPORT" == "xsyncv" ]; then
     awk '{ print "###", $0; } $1 == "sync_match" { correct += $2; missing += 3 - $2; incorrect += $3-$2; print "correct:", correct, "missing:", missing, "incorrect:", incorrect; }'
+  elif [ "x$AWM_REPORT" == "xtruncv" ]; then
+    awk ' {
+            print "###", $0;
+          }
+          $2 == "match_count" {
+            if (!n[$1])
+              {
+                n[$1]   = 0;
+                bad[$1] = 0;
+              }
+            if ($3 == 0)
+              bad[$1]++;
+            n[$1]++;
+          }
+          END {
+            for (trunc in n) {
+              print trunc, bad[trunc], n[trunc], bad[trunc] * 100.0 / n[trunc];
+            }
+          }'
   else
     echo "unknown report $AWM_REPORT" >&2
     exit 1

src/convcode.cc

@@ -22,11 +22,14 @@ parity (unsigned int v)
   return p;
 }
 
-// rate 1/6 code generator poynomial from "In search of a 2dB Coding Gain", Yuen and Vo
-// minimum free distance 56
-constexpr  unsigned int rate        = 6;
-constexpr  unsigned int order       = 15;
-constexpr  auto         generators  = std::array<unsigned,6> { 046321, 051271, 070535, 063667, 073277, 076531 };
+constexpr  auto         ab_generators = std::array<unsigned,12>
+  {
+    066561, 075211, 071545, 054435, 063635, 052475,
+    063543, 075307, 052547, 045627, 067657, 051757
+  };
+
+constexpr  unsigned int ab_rate       = ab_generators.size();
+constexpr  unsigned int order         = 15;
 
 /*
 constexpr  unsigned int order       = 9;
@@ -43,14 +46,45 @@ constexpr  unsigned int state_count = (1 << order);
 constexpr  unsigned int state_mask  = (1 << order) - 1;
 
 size_t
-conv_code_size (size_t msg_size)
+conv_code_size (ConvBlockType block_type, size_t msg_size)
+{
+  switch (block_type)
+    {
+      case ConvBlockType::a:
+      case ConvBlockType::b:  return (msg_size + order) * ab_rate / 2;
+      case ConvBlockType::ab: return (msg_size + order) * ab_rate;
+      default:                assert (false);
+    }
+}
+
+vector<unsigned>
+get_block_type_generators (ConvBlockType block_type)
 {
-  return (msg_size + order) * rate;
+  vector<unsigned> generators;
+
+  if (block_type == ConvBlockType::a)
+    {
+      for (unsigned int i = 0; i < ab_rate / 2; i++)
+        generators.push_back (ab_generators[i * 2]);
+    }
+  else if (block_type == ConvBlockType::b)
+    {
+      for (unsigned int i = 0; i < ab_rate / 2; i++)
+        generators.push_back (ab_generators[i * 2 + 1]);
+    }
+  else
+    {
+      assert (block_type == ConvBlockType::ab);
+      generators.assign (ab_generators.begin(), ab_generators.end());
+    }
+  return generators;
 }
 
 vector<int>
-conv_encode (const vector<int>& in_bits)
+conv_encode (ConvBlockType block_type, const vector<int>& in_bits)
 {
+  auto generators = get_block_type_generators (block_type);
+
   vector<int> out_vec;
   vector<int> vec = in_bits;
 
@@ -75,8 +109,10 @@ conv_encode (const vector<int>& in_bits)
 
 /* decode using viterbi algorithm */
 vector<int>
-conv_decode_soft (const vector<float>& coded_bits, float *error_out)
+conv_decode_soft (ConvBlockType block_type, const vector<float>& coded_bits, float *error_out)
 {
+  auto generators = get_block_type_generators (block_type);
+  unsigned int rate = generators.size();
   vector<int> decoded_bits;
 
   assert (coded_bits.size() % rate == 0);
@@ -121,7 +157,7 @@ conv_decode_soft (const vector<float>& coded_bits, float *error_out)
                   float delta = old_table[state].delta;
                   int   sbit_pos = new_state * rate;
 
-                  for (size_t p = 0; p < generators.size(); p++)
+                  for (size_t p = 0; p < rate; p++)
                     {
                       const float cbit = coded_bits[i + p];
                       const float sbit = state2bits[sbit_pos + p];
@@ -160,7 +196,7 @@ conv_decode_soft (const vector<float>& coded_bits, float *error_out)
 }
 
 vector<int>
-conv_decode_hard (const vector<int>& coded_bits)
+conv_decode_hard (ConvBlockType block_type, const vector<int>& coded_bits)
 {
   /* for the final application, we always want soft decoding, so we don't
    * special case hard decoding here, so this will be a little slower than
@@ -170,5 +206,24 @@ conv_decode_hard (const vector<int>& coded_bits)
   for (auto b : coded_bits)
     soft_bits.push_back (b ? 1.0f : 0.0f);
 
-  return conv_decode_soft (soft_bits);
+  return conv_decode_soft (block_type, soft_bits);
+}
+
+void
+conv_print_table (ConvBlockType block_type)
+{
+  vector<int> bits (100);
+  bits[0] = 1;
+
+  vector<int> out_bits = conv_encode (block_type, bits);
+
+  auto generators = get_block_type_generators (block_type);
+  unsigned int rate = generators.size();
+
+  for (unsigned int r = 0; r < rate; r++)
+    {
+      for (unsigned int i = 0; i < order; i++)
+        printf ("%s%d", i == 0 ? "" : " ", out_bits[i * rate + r]);
+      printf ("\n");
+    }
 }

src/convcode.hh

@@ -4,9 +4,13 @@
 #include <vector>
 #include <string>
 
-size_t           conv_code_size (size_t msg_size);
-std::vector<int> conv_encode (const std::vector<int>& in_bits);
-std::vector<int> conv_decode_hard (const std::vector<int>& coded_bits);
-std::vector<int> conv_decode_soft (const std::vector<float>& coded_bits, float *error_out = nullptr);
+enum class ConvBlockType { a, b, ab };
+
+size_t           conv_code_size (ConvBlockType block_type, size_t msg_size);
+std::vector<int> conv_encode (ConvBlockType block_type, const std::vector<int>& in_bits);
+std::vector<int> conv_decode_hard (ConvBlockType block_type, const std::vector<int>& coded_bits);
+std::vector<int> conv_decode_soft (ConvBlockType block_type, const std::vector<float>& coded_bits, float *error_out = nullptr);
+
+void             conv_print_table (ConvBlockType block_type);
 
 #endif /* AUDIOWMARK_CONV_CODE_HH */

src/gen-sync-adoc.sh

@@ -4,10 +4,9 @@ echo ".sync-codec-resistence"
 echo '[frame="topbot",options="header",cols="<2,6*>1"]'
 echo '|=========================='
 echo -n "| "
-for D in $(seq 10 -1 5)
+for STRENGTH in $(seq 10 -1 5)
 do
-  DELTA=$(printf "0.0%02d\n" $D)
-  echo -n "| $DELTA"
+  echo -n "| $STRENGTH"
 done
 echo
 for TEST in mp3 double-mp3 ogg
@@ -22,10 +21,9 @@ do
     echo "error: bad TEST $TEST ???"
     exit 1
   fi
-  for D in $(seq 10 -1 5)
+  for STRENGTH in $(seq 10 -1 5)
   do
-    DELTA=$(printf "0.0%02d\n" $D)
-    cat $DELTA-$TEST-* | awk '{bad += $1; n += $2} END {if (n==0) n=1;fer=100.0*bad/n; bold=fer>0?"*":" ";printf ("| %s%.2f%s", bold, fer, bold)}'
+    cat $STRENGTH-$TEST-* | grep -v '^#' | awk '{bad += $1; n += $2} END {if (n==0) n=1;fer=100.0*bad/n; bold=fer>0?"*":" ";printf ("| %s%.2f%s", bold, fer, bold)}'
   done
   echo
 done

src/gen-sync-mk.sh

 #!/bin/bash
 
-DELTA_RANGE="0.005 0.006 0.007 0.008 0.009 0.010"
+STRENGTH_RANGE=$(seq 5 10)
 SEEDS="$(seq 0 19)"
 
 echo -n "all: "
 for SEED in $SEEDS
 do
-  for DELTA in $DELTA_RANGE
+  for STRENGTH in $STRENGTH_RANGE
   do
-    echo -n "$DELTA-ogg-$SEED $DELTA-mp3-$SEED $DELTA-double-mp3-$SEED "
+    echo -n "$STRENGTH-ogg-$SEED $STRENGTH-mp3-$SEED $STRENGTH-double-mp3-$SEED "
   done
 done
 
@@ -17,23 +17,23 @@ echo
 
 for SEED in $SEEDS
 do
-  for DELTA in $DELTA_RANGE
+  for STRENGTH in $STRENGTH_RANGE
   do
-    FILE="$DELTA-ogg-$SEED"
+    FILE="$STRENGTH-ogg-$SEED"
     echo "$FILE:"
-    echo -e "\t( cd ..; AWM_RAND_PATTERN=1 AWM_RAND_CUT=1 AWM_SET=huge2 AWM_PARAMS='--water-delta $DELTA' AWM_REPORT=fer AWM_SEEDS=$SEED AWM_FILE='t-$FILE' ber-test.sh ogg 128 ) >x$FILE"
+    echo -e "\t( cd ..; AWM_RAND_PATTERN=1 AWM_RAND_CUT=1 AWM_SET=huge2 AWM_PARAMS='--strength $STRENGTH' AWM_REPORT=ferv AWM_SEEDS=$SEED AWM_FILE='t-$FILE' ber-test.sh ogg 128 ) >x$FILE"
     echo -e "\tmv x$FILE $FILE"
     echo
 
-    FILE="$DELTA-mp3-$SEED"
+    FILE="$STRENGTH-mp3-$SEED"
     echo "$FILE:"
-    echo -e "\t( cd ..; AWM_RAND_PATTERN=1 AWM_RAND_CUT=1 AWM_SET=huge2 AWM_PARAMS='--water-delta $DELTA' AWM_REPORT=fer AWM_SEEDS=$SEED AWM_FILE='t-$FILE' ber-test.sh mp3 128 ) >x$FILE"
+    echo -e "\t( cd ..; AWM_RAND_PATTERN=1 AWM_RAND_CUT=1 AWM_SET=huge2 AWM_PARAMS='--strength $STRENGTH' AWM_REPORT=ferv AWM_SEEDS=$SEED AWM_FILE='t-$FILE' ber-test.sh mp3 128 ) >x$FILE"
     echo -e "\tmv x$FILE $FILE"
     echo
 
-    FILE="$DELTA-double-mp3-$SEED"
+    FILE="$STRENGTH-double-mp3-$SEED"
     echo "$FILE:"
-    echo -e "\t( cd ..; AWM_RAND_PATTERN=1 AWM_RAND_CUT=1 AWM_SET=huge2 AWM_PARAMS='--water-delta $DELTA' AWM_REPORT=fer AWM_SEEDS=$SEED AWM_FILE='t-$FILE' ber-test.sh double-mp3 128 ) >x$FILE"
+    echo -e "\t( cd ..; AWM_RAND_PATTERN=1 AWM_RAND_CUT=1 AWM_SET=huge2 AWM_PARAMS='--strength $STRENGTH' AWM_REPORT=ferv AWM_SEEDS=$SEED AWM_FILE='t-$FILE' ber-test.sh double-mp3 128 ) >x$FILE"
     echo -e "\tmv x$FILE $FILE"
     echo
   done

src/gen-trunc-adoc.sh

+#!/bin/bash
+
+TRUNCATE="60 110 245"
+
+for TRUNC in 60 110 245
+do
+echo ".sync-codec-resistence$TRUNC"
+echo '[frame="topbot",options="header",cols="<2,6*>1"]'
+echo '|=========================='
+echo -n "| "
+for STRENGTH in $(seq 10 -1 5)
+do
+  echo -n "| $STRENGTH"
+done
+echo
+for TEST in mp3 double-mp3 ogg
+do
+  if [ $TEST == mp3 ]; then
+    echo -n "| mp3 128kbit/s"
+  elif [ $TEST == double-mp3 ]; then
+    echo -n "| double mp3 128kbit/s"
+  elif [ $TEST == ogg ]; then
+    echo -n "| ogg 128kbit/s"
+  else
+    echo "error: bad TEST $TEST ???"
+    exit 1
+  fi
+  for STRENGTH in $(seq 10 -1 5)
+  do
+    cat $STRENGTH-$TEST-* | grep -v '^#' | grep ^$TRUNC | awk '{bad += $2; n += $3} END {if (n==0) n=1;fer=100.0*bad/n; bold=fer>0?"*":" ";printf ("| %s%.2f%s", bold, fer, bold)}'
+  done
+  echo
+done
+echo
+echo '|=========================='
+done

src/gen-trunc-mk.sh

+#!/bin/bash
+
+STRENGTH_RANGE=$(seq 5 10)
+SEEDS="$(seq 0 19)"
+TRUNCATE="60 110 245"
+
+echo -n "all: "
+for SEED in $SEEDS
+do
+  for STRENGTH in $STRENGTH_RANGE
+  do
+    echo -n "$STRENGTH-ogg-$SEED $STRENGTH-mp3-$SEED $STRENGTH-double-mp3-$SEED "
+  done
+done
+
+echo
+echo
+
+for SEED in $SEEDS
+do
+  for STRENGTH in $STRENGTH_RANGE
+  do
+    FILE="$STRENGTH-ogg-$SEED"
+    echo "$FILE:"
+    echo -e "\t( cd ..; AWM_RAND_PATTERN=1 AWM_RAND_CUT=1 AWM_SET=huge2 AWM_PARAMS='--strength $STRENGTH' AWM_TRUNCATE='$TRUNCATE' AWM_SEEDS=$SEED AWM_FILE='t-$FILE' ber-test.sh ogg 128 ) >x$FILE"
+    echo -e "\tmv x$FILE $FILE"
+    echo
+
+    FILE="$STRENGTH-mp3-$SEED"
+    echo "$FILE:"
+    echo -e "\t( cd ..; AWM_RAND_PATTERN=1 AWM_RAND_CUT=1 AWM_SET=huge2 AWM_PARAMS='--strength $STRENGTH' AWM_TRUNCATE='$TRUNCATE' AWM_SEEDS=$SEED AWM_FILE='t-$FILE' ber-test.sh mp3 128 ) >x$FILE"
+    echo -e "\tmv x$FILE $FILE"
+    echo
+
+    FILE="$STRENGTH-double-mp3-$SEED"
+    echo "$FILE:"
+    echo -e "\t( cd ..; AWM_RAND_PATTERN=1 AWM_RAND_CUT=1 AWM_SET=huge2 AWM_PARAMS='--strength $STRENGTH' AWM_TRUNCATE='$TRUNCATE' AWM_SEEDS=$SEED AWM_FILE='t-$FILE' ber-test.sh double-mp3 128 ) >x$FILE"
+    echo -e "\tmv x$FILE $FILE"
+    echo
+  done
+done

src/random.cc

@@ -10,6 +10,31 @@ using std::vector;
 using std::regex;
 using std::regex_match;
 
+static void
+gcrypt_init()
+{
+  static bool init_ok = false;
+
+  if (!init_ok)
+    {
+      /* version check: start libgcrypt initialization */
+      if (!gcry_check_version (GCRYPT_VERSION))
+        {
+          fprintf (stderr, "audiowmark: libgcrypt version mismatch\n");
+          exit (1);
+        }
+
+      /* disable secure memory (assume we run in a controlled environment) */
+      gcry_control (GCRYCTL_DISABLE_SECMEM, 0);
+
+      /* tell libgcrypt that initialization has completed */
+      gcry_control (GCRYCTL_INITIALIZATION_FINISHED, 0);
+
+      init_ok = true;
+    }
+}
+
+
 static vector<unsigned char> aes_key (16); // 128 bits
 static constexpr auto        GCRY_CIPHER = GCRY_CIPHER_AES128;
 
@@ -55,6 +80,8 @@ print (const string& label, const vector<unsigned char>& data)
 
 Random::Random (uint64_t seed, Stream stream)
 {
+  gcrypt_init();
+
   vector<unsigned char> ctr = get_start_counter (seed, stream);
 
   // print ("CTR", ctr);
@@ -201,6 +228,8 @@ Random::load_global_key (const string& key_file)
 string
 Random::gen_key()
 {
+  gcrypt_init();
+
   vector<unsigned char> key (16);
   gcry_randomize (&key[0], 16, /* long term key material strength */ GCRY_VERY_STRONG_RANDOM);
   return vec_to_hex_str (key);

src/random.hh

@@ -15,7 +15,8 @@ public:
     sync_up_down = 2,
     pad_up_down = 3,
     mix = 4,
-    bit_order = 5
+    bit_order = 5,
+    frame_position = 6
   };
 private:
   gcry_cipher_hd_t           aes_ctr_cipher;

src/testconvcode.cc

@@ -34,10 +34,36 @@ generate_error_vector (size_t n, int errors)
     }
   return ev;
 }
+
+static bool
+no_case_equal (const string& s1, const string& s2)
+{
+  if (s1.size() != s2.size())
+    return false;
+
+  return std::equal (s1.begin(), s1.end(), s2.begin(),
+                     [] (char c1, char c2) -> bool { return tolower (c1) == tolower (c2);});
+}
+
 int
 main (int argc, char **argv)
 {
-  if (argc == 1)
+  string btype = (argc > 1) ? argv[1] : "";
+  ConvBlockType block_type;
+
+  if (no_case_equal (btype, "A"))
+    block_type = ConvBlockType::a;
+  else if (no_case_equal (btype, "B"))
+    block_type = ConvBlockType::b;
+  else if (no_case_equal (btype, "AB"))
+    block_type = ConvBlockType::ab;
+  else
+    {
+      printf ("first argument must be A, B, or AB\n");
+      return 1;
+    }
+
+  if (argc == 2)
     {
       vector<int> in_bits = bit_str_to_vec ("80f12381");
 
@@ -46,16 +72,16 @@ main (int argc, char **argv)
         printf ("%d", b);
       printf ("\n");
 
-      vector<int> coded_bits = conv_encode (in_bits);
+      vector<int> coded_bits = conv_encode (block_type, in_bits);
       printf ("coded vector (n=%zd): ", coded_bits.size());
       for (auto b : coded_bits)
         printf ("%d", b);
       printf ("\n");
       printf ("coded hex: %s\n", bit_vec_to_str (coded_bits).c_str());
 
-      assert (coded_bits.size() == conv_code_size (in_bits.size()));
+      assert (coded_bits.size() == conv_code_size (block_type, in_bits.size()));
 
-      vector<int> decoded_bits = conv_decode_hard (coded_bits);
+      vector<int> decoded_bits = conv_decode_hard (block_type, coded_bits);
       printf ("output vector (k=%zd): ", decoded_bits.size());
       for (auto b : decoded_bits)
         printf ("%d", b);
@@ -68,9 +94,9 @@ main (int argc, char **argv)
           errors++;
       printf ("decoding errors: %d\n", errors);
     }
-  if (argc == 2 && string (argv[1]) == "error")
+  if (argc == 3 && string (argv[2]) == "error")
     {
-      size_t max_bit_errors = conv_code_size (128) * 0.5;
+      size_t max_bit_errors = conv_code_size (block_type, 128) * 0.5;
 
       for (size_t bit_errors = 0; bit_errors < max_bit_errors; bit_errors++)
         {
@@ -84,14 +110,14 @@ main (int argc, char **argv)
               while (in_bits.size() != 128)
                 in_bits.push_back (rand() & 1);
 
-              vector<int> coded_bits = conv_encode (in_bits);
+              vector<int> coded_bits = conv_encode (block_type, in_bits);
               coded_bit_count = coded_bits.size();
 
               vector<int> error_bits = generate_error_vector (coded_bits.size(), bit_errors);
               for (size_t pos = 0; pos < coded_bits.size(); pos++)
                 coded_bits[pos] ^= error_bits[pos];
 
-              vector<int> decoded_bits = conv_decode_hard (coded_bits);
+              vector<int> decoded_bits = conv_decode_hard (block_type, coded_bits);
 
               assert (decoded_bits.size() == 128);
 
@@ -105,7 +131,7 @@ main (int argc, char **argv)
           printf ("%f %f\n", (100.0 * bit_errors) / coded_bit_count, (100.0 * bad_decode) / test_size);
         }
     }
-  if (argc == 2 && string (argv[1]) == "soft-error")
+  if (argc == 3 && string (argv[2]) == "soft-error")
     {
       for (double stddev = 0; stddev < 1.5; stddev += 0.01)
         {
@@ -120,7 +146,7 @@ main (int argc, char **argv)
               while (in_bits.size() != 128)
                 in_bits.push_back (rand() & 1);
 
-              vector<int> coded_bits = conv_encode (in_bits);
+              vector<int> coded_bits = conv_encode (block_type, in_bits);
               coded_bit_count = coded_bits.size();
 
               std::default_random_engine generator;
@@ -130,7 +156,7 @@ main (int argc, char **argv)
               for (auto b : coded_bits)
                 recv_bits.push_back (b + dist (generator));
 
-              vector<int> decoded_bits1 = conv_decode_soft (recv_bits);
+              vector<int> decoded_bits1 = conv_decode_soft (block_type, recv_bits);
 
               vector<int> recv_hard_bits;
               for (auto b : recv_bits)
@@ -139,7 +165,7 @@ main (int argc, char **argv)
               for (size_t x = 0; x < recv_hard_bits.size(); x++)
                 local_be += coded_bits[x] ^ recv_hard_bits[x];
 
-              vector<int> decoded_bits2 = conv_decode_hard (recv_hard_bits);
+              vector<int> decoded_bits2 = conv_decode_hard (block_type, recv_hard_bits);
 
               assert (decoded_bits1.size() == 128);
               assert (decoded_bits2.size() == 128);
@@ -161,7 +187,7 @@ main (int argc, char **argv)
           printf ("%f %f %f\n", double (100 * local_be) / test_size / coded_bit_count, (100.0 * bad_decode1) / test_size, (100.0 * bad_decode2) / test_size);
         }
     }
-  if (argc == 2 && string (argv[1]) == "perf")
+  if (argc == 3 && string (argv[2]) == "perf")
     {
       vector<int> in_bits;
       while (in_bits.size() != 128)
@@ -171,9 +197,11 @@ main (int argc, char **argv)
       const size_t runs = 20;
       for (size_t i = 0; i < runs; i++)
         {
-          vector<int> out_bits = conv_decode_hard (conv_encode (in_bits));
+          vector<int> out_bits = conv_decode_hard (block_type, conv_encode (block_type, in_bits));
           assert (out_bits == in_bits);
         }
       printf ("%.1f ms/block\n", (gettime() - start_t) / runs * 1000.0);
     }
+  if (argc == 3 && string (argv[2]) == "table")
+    conv_print_table (block_type);
 }