PyTorch is an extensive tool for deep learning. Let’s discuss how the central data structure the Tensor is designed. What are the pitfalls of a simple operation
Join the DZone community and get the full member experience. It is a reasonable thing to expect n-dimensional tensor to have a possibility to be reshaped. Reshape means to change the spatial size of a container that holds underlying data. One can create any n-dimensional tensor that wraps a numerical array as long as the product of dimensions stays equal to the number of the array’s elements. The same holds for reshaping. It’s possible to convert the existing container to a different spatial size while it preserves the total number of elements. This is quite intuitive. But, the experienced users might notice a couple of not obvious things. The answer to the first question is that reshaped tensor sometimes triggers copying of underlying data and sometimes doesn’t. The documentation says the following about reshape method: This quote introduces three things: the view of a tensor, the stride of a tensor, and contiguous tensors. Let’s talk about them in order. Stride is a tuple of integers each of which represents a jump needed to perform on the underlying data to go to the next element within the current dimension. The tensor’s underlying data is just a one-dimensional physical array, that is stored sequentially in the memory. If we have, for example, an arbitrary 3-dimensional container, then the elements at indices (x,y,z) and at (x+1,y,z) live at a distance of size stride(0) within the underlying data. See the code below: Using all the stride entries we can translate any given n-dimensional index to the offset within a physical array by ourselves. Here’s the graphical interpretation of a stride. This image is cherry-picked from ezyang’s blog, one of the core PyTorch maintainers.
