postcard/ser/flavors.rs
1//! # Serialization Flavors
2//!
3//! "Flavors" in `postcard` are used as modifiers to the serialization or deserialization
4//! process. Flavors typically modify one or both of the following:
5//!
6//! 1. The output medium of the serialization, e.g. whether the data is serialized to a `[u8]` slice, or a `heapless::Vec`.
7//! 2. The format of the serialization, such as encoding the serialized output in a COBS format, performing CRC32 checksumming while serializing, etc.
8//!
9//! Flavors are implemented using the [`Flavor`] trait, which acts as a "middleware" for receiving the bytes as serialized by `serde`.
10//! Multiple flavors may be combined to obtain a desired combination of behavior and storage.
11//! When flavors are combined, it is expected that the storage flavor (such as `Slice` or `HVec`) is the innermost flavor.
12//!
13//! Custom flavors may be defined by users of the `postcard` crate, however some commonly useful flavors have been provided in
14//! this module. If you think your custom flavor would be useful to others, PRs adding flavors are very welcome!
15//!
16//! ## Usability
17//!
18//! Flavors may not always be convenient to use directly, as they may expose some implementation details of how the
19//! inner workings of the flavor behaves. It is typical to provide a convenience method for using a flavor, to prevent
20//! the user from having to specify generic parameters, setting correct initialization values, or handling the output of
21//! the flavor correctly. See `postcard::to_vec()` for an example of this.
22//!
23//! It is recommended to use the [`serialize_with_flavor()`](../fn.serialize_with_flavor.html) method for serialization. See it's documentation for information
24//! regarding usage and generic type parameters.
25//!
26//! ## When to use (multiple) flavors
27//!
28//! Combining flavors are nice for convenience, as they perform potentially multiple steps of
29//! serialization at one time.
30//!
31//! This can often be more memory efficient, as intermediate buffers are not typically required.
32//!
33//! ## When NOT to use (multiple) flavors
34//!
35//! The downside of passing serialization through multiple steps is that it is typically slower than
36//! performing each step serially. Said simply, "cobs encoding while serializing" is often slower
37//! than "serialize then cobs encode", due to the ability to handle longer "runs" of data in each
38//! stage. The downside is that if these stages can not be performed in-place on the buffer, you
39//! will need additional buffers for each stage.
40//!
41//! ## Examples
42//!
43//! ### Using a single flavor
44//!
45//! In the first example, we use the `Slice` flavor, to store the serialized output into a mutable `[u8]` slice.
46//! No other modification is made to the serialization process.
47//!
48//! ```rust
49//! use postcard::{
50//! serialize_with_flavor,
51//! ser_flavors::Slice,
52//! };
53//!
54//! let mut buf = [0u8; 32];
55//!
56//! let data: &[u8] = &[0x01, 0x00, 0x20, 0x30];
57//! let buffer = &mut [0u8; 32];
58//! let res = serialize_with_flavor::<[u8], Slice, &mut [u8]>(
59//! data,
60//! Slice::new(buffer)
61//! ).unwrap();
62//!
63//! assert_eq!(res, &[0x04, 0x01, 0x00, 0x20, 0x30]);
64//! ```
65//!
66//! ### Using combined flavors
67//!
68//! In the second example, we mix `Slice` with `Cobs`, to cobs encode the output while
69//! the data is serialized. Notice how `Slice` (the storage flavor) is the innermost flavor used.
70//!
71//! ```rust
72//! use postcard::{
73//! serialize_with_flavor,
74//! ser_flavors::{Cobs, Slice},
75//! };
76//!
77//! let mut buf = [0u8; 32];
78//!
79//! let data: &[u8] = &[0x01, 0x00, 0x20, 0x30];
80//! let buffer = &mut [0u8; 32];
81//! let res = serialize_with_flavor::<[u8], Cobs<Slice>, &mut [u8]>(
82//! data,
83//! Cobs::try_new(Slice::new(buffer)).unwrap(),
84//! ).unwrap();
85//!
86//! assert_eq!(res, &[0x03, 0x04, 0x01, 0x03, 0x20, 0x30, 0x00]);
87//! ```
88
89use crate::error::{Error, Result};
90use cobs::{EncoderState, PushResult};
91use core::marker::PhantomData;
92use core::ops::Index;
93use core::ops::IndexMut;
94
95#[cfg(feature = "heapless")]
96pub use heapless_vec::*;
97
98#[cfg(feature = "use-std")]
99pub use std_vec::*;
100
101#[cfg(feature = "alloc")]
102pub use alloc_vec::*;
103
104#[cfg(feature = "alloc")]
105extern crate alloc;
106
107/// The serialization Flavor trait
108///
109/// This is used as the primary way to encode serialized data into some kind of buffer,
110/// or modify that data in a middleware style pattern.
111///
112/// See the module level docs for an example of how flavors are used.
113pub trait Flavor {
114 /// The `Output` type is what this storage "resolves" to when the serialization is complete,
115 /// such as a slice or a Vec of some sort.
116 type Output;
117
118 /// Override this method when you want to customize processing
119 /// multiple bytes at once, such as copying a slice to the output,
120 /// rather than iterating over one byte at a time.
121 #[inline]
122 fn try_extend(&mut self, data: &[u8]) -> Result<()> {
123 data.iter().try_for_each(|d| self.try_push(*d))
124 }
125
126 /// Push a single byte to be modified and/or stored.
127 fn try_push(&mut self, data: u8) -> Result<()>;
128
129 /// Finalize the serialization process.
130 fn finalize(self) -> Result<Self::Output>;
131}
132
133////////////////////////////////////////
134// Slice
135////////////////////////////////////////
136
137/// The `Slice` flavor is a storage flavor, storing the serialized (or otherwise modified) bytes into a plain
138/// `[u8]` slice. The `Slice` flavor resolves into a sub-slice of the original slice buffer.
139pub struct Slice<'a> {
140 start: *mut u8,
141 cursor: *mut u8,
142 end: *mut u8,
143 _pl: PhantomData<&'a [u8]>,
144}
145
146impl<'a> Slice<'a> {
147 /// Create a new `Slice` flavor from a given backing buffer
148 pub fn new(buf: &'a mut [u8]) -> Self {
149 let ptr = buf.as_mut_ptr();
150 Slice {
151 start: ptr,
152 cursor: ptr,
153 end: unsafe { ptr.add(buf.len()) },
154 _pl: PhantomData,
155 }
156 }
157}
158
159impl<'a> Flavor for Slice<'a> {
160 type Output = &'a mut [u8];
161
162 #[inline(always)]
163 fn try_push(&mut self, b: u8) -> Result<()> {
164 if self.cursor == self.end {
165 Err(Error::SerializeBufferFull)
166 } else {
167 unsafe {
168 self.cursor.write(b);
169 self.cursor = self.cursor.add(1);
170 }
171 Ok(())
172 }
173 }
174
175 #[inline(always)]
176 fn try_extend(&mut self, b: &[u8]) -> Result<()> {
177 let remain = (self.end as usize) - (self.cursor as usize);
178 let blen = b.len();
179 if blen > remain {
180 Err(Error::SerializeBufferFull)
181 } else {
182 unsafe {
183 core::ptr::copy_nonoverlapping(b.as_ptr(), self.cursor, blen);
184 self.cursor = self.cursor.add(blen);
185 }
186 Ok(())
187 }
188 }
189
190 fn finalize(self) -> Result<Self::Output> {
191 let used = (self.cursor as usize) - (self.start as usize);
192 let sli = unsafe { core::slice::from_raw_parts_mut(self.start, used) };
193 Ok(sli)
194 }
195}
196
197impl Index<usize> for Slice<'_> {
198 type Output = u8;
199
200 fn index(&self, idx: usize) -> &u8 {
201 let len = (self.end as usize) - (self.start as usize);
202 assert!(idx < len);
203 unsafe { &*self.start.add(idx) }
204 }
205}
206
207impl IndexMut<usize> for Slice<'_> {
208 fn index_mut(&mut self, idx: usize) -> &mut u8 {
209 let len = (self.end as usize) - (self.start as usize);
210 assert!(idx < len);
211 unsafe { &mut *self.start.add(idx) }
212 }
213}
214
215/// Wrapper over a [`core::iter::Extend<u8>`] that implements the flavor trait
216pub struct ExtendFlavor<T> {
217 iter: T,
218}
219
220impl<T> ExtendFlavor<T>
221where
222 T: core::iter::Extend<u8>,
223{
224 /// Create a new [`Self`] flavor from a given [`core::iter::Extend<u8>`]
225 pub fn new(iter: T) -> Self {
226 Self { iter }
227 }
228}
229
230impl<T> Flavor for ExtendFlavor<T>
231where
232 T: core::iter::Extend<u8>,
233{
234 type Output = T;
235
236 #[inline(always)]
237 fn try_push(&mut self, data: u8) -> Result<()> {
238 self.iter.extend([data]);
239 Ok(())
240 }
241
242 #[inline(always)]
243 fn try_extend(&mut self, b: &[u8]) -> Result<()> {
244 self.iter.extend(b.iter().copied());
245 Ok(())
246 }
247
248 fn finalize(self) -> Result<Self::Output> {
249 Ok(self.iter)
250 }
251}
252
253/// Support for the [`embedded-io`](crate::eio::embedded_io) traits
254#[cfg(any(feature = "embedded-io-04", feature = "embedded-io-06"))]
255pub mod eio {
256
257 use super::Flavor;
258 use crate::{Error, Result};
259
260 /// Wrapper over a [`embedded_io Write`](crate::eio::Write) that implements the flavor trait
261 pub struct WriteFlavor<T> {
262 writer: T,
263 }
264
265 impl<T> WriteFlavor<T>
266 where
267 T: crate::eio::Write,
268 {
269 /// Create a new [`Self`] flavor from a given [`embedded_io Write`](crate::eio::Write)
270 pub fn new(writer: T) -> Self {
271 Self { writer }
272 }
273 }
274
275 impl<T> Flavor for WriteFlavor<T>
276 where
277 T: crate::eio::Write,
278 {
279 type Output = T;
280
281 #[inline(always)]
282 fn try_push(&mut self, data: u8) -> Result<()> {
283 self.writer
284 .write_all(&[data])
285 .map_err(|_| Error::SerializeBufferFull)?;
286 Ok(())
287 }
288
289 #[inline(always)]
290 fn try_extend(&mut self, b: &[u8]) -> Result<()> {
291 self.writer
292 .write_all(b)
293 .map_err(|_| Error::SerializeBufferFull)?;
294 Ok(())
295 }
296
297 fn finalize(mut self) -> Result<Self::Output> {
298 self.writer
299 .flush()
300 .map_err(|_| Error::SerializeBufferFull)?;
301 Ok(self.writer)
302 }
303 }
304}
305
306/// Support for the [`std::io`] traits
307#[cfg(feature = "use-std")]
308pub mod io {
309
310 use super::Flavor;
311 use crate::{Error, Result};
312
313 /// Wrapper over a [`std::io::Write`] that implements the flavor trait
314 pub struct WriteFlavor<T> {
315 writer: T,
316 }
317
318 impl<T> WriteFlavor<T>
319 where
320 T: std::io::Write,
321 {
322 /// Create a new [`Self`] flavor from a given [`std::io::Write`]
323 pub fn new(writer: T) -> Self {
324 Self { writer }
325 }
326 }
327
328 impl<T> Flavor for WriteFlavor<T>
329 where
330 T: std::io::Write,
331 {
332 type Output = T;
333
334 #[inline(always)]
335 fn try_push(&mut self, data: u8) -> Result<()> {
336 self.writer
337 .write_all(&[data])
338 .map_err(|_| Error::SerializeBufferFull)?;
339 Ok(())
340 }
341
342 #[inline(always)]
343 fn try_extend(&mut self, b: &[u8]) -> Result<()> {
344 self.writer
345 .write_all(b)
346 .map_err(|_| Error::SerializeBufferFull)?;
347 Ok(())
348 }
349
350 fn finalize(mut self) -> Result<Self::Output> {
351 self.writer
352 .flush()
353 .map_err(|_| Error::SerializeBufferFull)?;
354 Ok(self.writer)
355 }
356 }
357}
358
359#[cfg(feature = "heapless")]
360mod heapless_vec {
361 use super::Flavor;
362 use super::Index;
363 use super::IndexMut;
364 use crate::{Error, Result};
365 use heapless::Vec;
366
367 ////////////////////////////////////////
368 // HVec
369 ////////////////////////////////////////
370
371 /// The `HVec` flavor is a wrapper type around a `heapless::Vec`. This is a stack
372 /// allocated data structure, with a fixed maximum size and variable amount of contents.
373 #[derive(Default)]
374 pub struct HVec<const B: usize> {
375 /// the contained data buffer
376 vec: Vec<u8, B>,
377 }
378
379 impl<const B: usize> HVec<B> {
380 /// Create a new, currently empty, [`heapless::Vec`] to be used for storing serialized
381 /// output data.
382 pub fn new() -> Self {
383 Self::default()
384 }
385 }
386
387 impl<const B: usize> Flavor for HVec<B> {
388 type Output = Vec<u8, B>;
389
390 #[inline(always)]
391 fn try_extend(&mut self, data: &[u8]) -> Result<()> {
392 self.vec
393 .extend_from_slice(data)
394 .map_err(|_| Error::SerializeBufferFull)
395 }
396
397 #[inline(always)]
398 fn try_push(&mut self, data: u8) -> Result<()> {
399 self.vec.push(data).map_err(|_| Error::SerializeBufferFull)
400 }
401
402 fn finalize(self) -> Result<Vec<u8, B>> {
403 Ok(self.vec)
404 }
405 }
406
407 impl<const B: usize> Index<usize> for HVec<B> {
408 type Output = u8;
409
410 fn index(&self, idx: usize) -> &u8 {
411 &self.vec[idx]
412 }
413 }
414
415 impl<const B: usize> IndexMut<usize> for HVec<B> {
416 fn index_mut(&mut self, idx: usize) -> &mut u8 {
417 &mut self.vec[idx]
418 }
419 }
420}
421
422#[cfg(feature = "use-std")]
423mod std_vec {
424 /// The `StdVec` flavor is a wrapper type around a `std::vec::Vec`.
425 ///
426 /// This type is only available when the (non-default) `use-std` feature is active
427 pub type StdVec = super::alloc_vec::AllocVec;
428}
429
430#[cfg(feature = "alloc")]
431mod alloc_vec {
432 extern crate alloc;
433 use super::Flavor;
434 use super::Index;
435 use super::IndexMut;
436 use crate::Result;
437 use alloc::vec::Vec;
438
439 /// The `AllocVec` flavor is a wrapper type around an [`alloc::vec::Vec`].
440 ///
441 /// This type is only available when the (non-default) `alloc` feature is active
442 #[derive(Default)]
443 pub struct AllocVec {
444 /// The vec to be used for serialization
445 vec: Vec<u8>,
446 }
447
448 impl AllocVec {
449 /// Create a new, currently empty, [`alloc::vec::Vec`] to be used for storing serialized
450 /// output data.
451 pub fn new() -> Self {
452 Self::default()
453 }
454 }
455
456 impl Flavor for AllocVec {
457 type Output = Vec<u8>;
458
459 #[inline(always)]
460 fn try_extend(&mut self, data: &[u8]) -> Result<()> {
461 self.vec.extend_from_slice(data);
462 Ok(())
463 }
464
465 #[inline(always)]
466 fn try_push(&mut self, data: u8) -> Result<()> {
467 self.vec.push(data);
468 Ok(())
469 }
470
471 fn finalize(self) -> Result<Self::Output> {
472 Ok(self.vec)
473 }
474 }
475
476 impl Index<usize> for AllocVec {
477 type Output = u8;
478
479 #[inline]
480 fn index(&self, idx: usize) -> &u8 {
481 &self.vec[idx]
482 }
483 }
484
485 impl IndexMut<usize> for AllocVec {
486 #[inline]
487 fn index_mut(&mut self, idx: usize) -> &mut u8 {
488 &mut self.vec[idx]
489 }
490 }
491}
492
493////////////////////////////////////////////////////////////////////////////////
494// Modification Flavors
495////////////////////////////////////////////////////////////////////////////////
496
497////////////////////////////////////////
498// COBS
499////////////////////////////////////////
500
501/// The `Cobs` flavor implements [Consistent Overhead Byte Stuffing] on
502/// the serialized data. The output of this flavor includes the termination/sentinel
503/// byte of `0x00`.
504///
505/// This protocol is useful when sending data over a serial interface without framing such as a UART
506///
507/// [Consistent Overhead Byte Stuffing]: https://en.wikipedia.org/wiki/Consistent_Overhead_Byte_Stuffing
508pub struct Cobs<B>
509where
510 B: Flavor + IndexMut<usize, Output = u8>,
511{
512 flav: B,
513 cobs: EncoderState,
514}
515
516impl<B> Cobs<B>
517where
518 B: Flavor + IndexMut<usize, Output = u8>,
519{
520 /// Create a new Cobs modifier Flavor. If there is insufficient space
521 /// to push the leading header byte, the method will return an Error
522 pub fn try_new(mut bee: B) -> Result<Self> {
523 bee.try_push(0).map_err(|_| Error::SerializeBufferFull)?;
524 Ok(Self {
525 flav: bee,
526 cobs: EncoderState::default(),
527 })
528 }
529}
530
531impl<B> Flavor for Cobs<B>
532where
533 B: Flavor + IndexMut<usize, Output = u8>,
534{
535 type Output = <B as Flavor>::Output;
536
537 #[inline(always)]
538 fn try_push(&mut self, data: u8) -> Result<()> {
539 use PushResult::*;
540 match self.cobs.push(data) {
541 AddSingle(n) => self.flav.try_push(n),
542 ModifyFromStartAndSkip((idx, mval)) => {
543 self.flav[idx] = mval;
544 self.flav.try_push(0)
545 }
546 ModifyFromStartAndPushAndSkip((idx, mval, nval)) => {
547 self.flav[idx] = mval;
548 self.flav.try_push(nval)?;
549 self.flav.try_push(0)
550 }
551 }
552 }
553
554 fn finalize(mut self) -> Result<Self::Output> {
555 let (idx, mval) = self.cobs.finalize();
556 self.flav[idx] = mval;
557 self.flav.try_push(0)?;
558 self.flav.finalize()
559 }
560}
561
562////////////////////////////////////////
563// CRC
564////////////////////////////////////////
565
566/// This Cyclic Redundancy Check flavor applies [the CRC crate's `Algorithm`](https://docs.rs/crc/latest/crc/struct.Algorithm.html) struct on
567/// the serialized data.
568///
569/// The output of this flavor receives the CRC appended to the bytes.
570/// CRCs are used for error detection when reading data back.
571/// Requires the `crc` feature.
572///
573/// More on CRCs: <https://en.wikipedia.org/wiki/Cyclic_redundancy_check>.
574#[cfg(feature = "use-crc")]
575#[cfg_attr(docsrs, doc(cfg(feature = "use-crc")))]
576pub mod crc {
577 use crc::Digest;
578 use crc::Width;
579 use serde::Serialize;
580
581 #[cfg(feature = "alloc")]
582 use super::alloc;
583 use super::Flavor;
584 use super::Slice;
585
586 use crate::serialize_with_flavor;
587 use crate::Result;
588 use paste::paste;
589
590 /// Manages CRC modifications as a flavor.
591 pub struct CrcModifier<'a, B, W>
592 where
593 B: Flavor,
594 W: Width,
595 {
596 flav: B,
597 digest: Digest<'a, W>,
598 }
599
600 impl<'a, B, W> CrcModifier<'a, B, W>
601 where
602 B: Flavor,
603 W: Width,
604 {
605 /// Create a new CRC modifier Flavor.
606 pub fn new(bee: B, digest: Digest<'a, W>) -> Self {
607 Self { flav: bee, digest }
608 }
609 }
610
611 macro_rules! impl_flavor {
612 ($( $int:ty ),*) => {
613 $(
614 paste! {
615 impl<'a, B> Flavor for CrcModifier<'a, B, $int>
616 where
617 B: Flavor,
618 {
619 type Output = <B as Flavor>::Output;
620
621 #[inline(always)]
622 fn try_push(&mut self, data: u8) -> Result<()> {
623 self.digest.update(&[data]);
624 self.flav.try_push(data)
625 }
626
627 fn finalize(mut self) -> Result<Self::Output> {
628 let crc = self.digest.finalize();
629 for byte in crc.to_le_bytes() {
630 self.flav.try_push(byte)?;
631 }
632 self.flav.finalize()
633 }
634 }
635
636 /// Serialize a `T` to the given slice, with the resulting slice containing
637 /// data followed by a CRC. The CRC bytes are included in the output buffer.
638 ///
639 /// When successful, this function returns the slice containing the
640 /// serialized and encoded message.
641 pub fn [<to_slice_ $int>]<'a, T>(
642 value: &T,
643 buf: &'a mut [u8],
644 digest: Digest<'_, $int>,
645 ) -> Result<&'a mut [u8]>
646 where
647 T: Serialize + ?Sized,
648 {
649 serialize_with_flavor(value, CrcModifier::new(Slice::new(buf), digest))
650 }
651
652 /// Serialize a `T` to a `heapless::Vec<u8>`, with the `Vec` containing
653 /// data followed by a CRC. The CRC bytes are included in the output `Vec`.
654 #[cfg(feature = "heapless")]
655 #[cfg_attr(docsrs, doc(cfg(feature = "heapless")))]
656 pub fn [<to_vec_ $int>]<T, const B: usize>(
657 value: &T,
658 digest: Digest<'_, $int>,
659 ) -> Result<heapless::Vec<u8, B>>
660 where
661 T: Serialize + ?Sized,
662 {
663 use super::HVec;
664
665 serialize_with_flavor(value, CrcModifier::new(HVec::default(), digest))
666 }
667
668 /// Serialize a `T` to a `heapless::Vec<u8>`, with the `Vec` containing
669 /// data followed by a CRC. The CRC bytes are included in the output `Vec`.
670 #[cfg(feature = "alloc")]
671 #[cfg_attr(docsrs, doc(cfg(feature = "alloc")))]
672 pub fn [<to_allocvec_ $int>]<T>(value: &T, digest: Digest<'_, $int>) -> Result<alloc::vec::Vec<u8>>
673 where
674 T: Serialize + ?Sized,
675 {
676 use super::AllocVec;
677
678 serialize_with_flavor(value, CrcModifier::new(AllocVec::new(), digest))
679 }
680 }
681 )*
682 };
683 }
684
685 impl_flavor![u8, u16, u32, u64, u128];
686}
687
688/// The `Size` flavor is a measurement flavor, which accumulates the number of bytes needed to
689/// serialize the data.
690///
691/// ```
692/// use postcard::{serialize_with_flavor, ser_flavors};
693///
694/// let value = false;
695/// let size = serialize_with_flavor(&value, ser_flavors::Size::default()).unwrap();
696///
697/// assert_eq!(size, 1);
698/// ```
699#[derive(Default)]
700pub struct Size {
701 size: usize,
702}
703
704impl Flavor for Size {
705 type Output = usize;
706
707 #[inline(always)]
708 fn try_push(&mut self, _b: u8) -> Result<()> {
709 self.size += 1;
710 Ok(())
711 }
712
713 #[inline(always)]
714 fn try_extend(&mut self, b: &[u8]) -> Result<()> {
715 self.size += b.len();
716 Ok(())
717 }
718
719 fn finalize(self) -> Result<Self::Output> {
720 Ok(self.size)
721 }
722}