Spatial Invariance
We essentially scan the waves with a motion detector and assign a score to each run of waves, indicating the likelihood the subject is lifting, grasping or holding.
CNNs systematize this idea of spatial invariance, exploiting it to learn useful representations with fewer parameters.
What does it mean by “Convolutional”? and why should we care?
While previously, we might have required billions of parameters to represent just a single layer in an data-processing network, we now typically need just a few hundred, without altering the dimensionality of either the inputs or the hidden representations. The price paid for this drastic reduction in parameters is that our features are now translation invariant and that our layer can only incorporate local information, when determining the value of each hidden activation. All learning depends on imposing inductive bias. When that bias agrees with reality, we get sample-efficient models that generalize well to unseen data. But of course, if those biases do not agree with reality, e.g., if data turned out not to be translation invariant, our models might struggle even to fit our training data.
Mathematical representation
we measure the overlap between f and g when one function is “flipped” and shifted by x.
The model specs:
we implemented a CNN with 32 output nodes and one Leaky RELU with optimum parameters for dimensionality reduction without affecting the performance of the model
self.net = nn.Sequential( nn.Conv1d(opt.in_channels, opt.in_channels, 5, padding=2), nn.Conv1d(self.hidden, self.hidden, 16, stride=16), nn.LeakyReLU(0.1), nn.Conv1d(self.hidden, self.hidden, 7, padding=3), )
Receiver operating characteristic (ROC)
is a graphical plot that illustrates the diagnostic ability of the CNN as its parameters are varied.
