![]() ![]() In other words: 3 x 3 does not equal 16 but 9. If you do: a.view(3, 3), it will raise a RuntimeError because shape (3 x 3) is invalid for input with 16 elements. For example: a = torch.arange(1, 17) # a's shape is (16,)Īssuming that -1 is not one of the parameters, when you multiply them together, the result must be equal to the number of elements in the tensor. The view method returns a tensor with the same data as the self tensor (which means that the returned tensor has the same number of elements), but with a different shape. Let's do some examples, from simpler to more difficult. Hence t1.view(3,2) in our example would be equivalent to t1.view(3,-1) or t1.view(-1,2). It is a convention taken from numpy.reshape(). the quotient of the original product by the new product). 1 is a PyTorch alias for "infer this dimension given the others have all been specified" (i.e. the old and new tensor have the same volume). Note this implicitly assumes that the new and old dimensions have the same product (i.e. (by modifying the shape and stride attributes). View(dim1,dim2.) returns a view of the same underlying information, but reshaped to a tensor of shape dim1 x dim2 x. stride states how many steps you need to take in memory til you reach the next element in each dimension.shape states how long each dimension is.Here you see PyTorch makes a tensor by converting an underlying block of contiguous memory into a matrix-like object by adding a shape and stride attribute: the right-hand tensor (shape (3,2)) can be computed from the left-hand one with t2 = t1.view(3,2) So you tell PyTorch to reshape the tensor you obtained to have specific number of columns and tell it to decide the number of rows by itself.Į.g. You have to flatten this to give it to the fully connected layer. After the line x = self.pool(F.relu(nv2(x))) in the forward function, you will have a 16 depth feature map. This can be seen in this model definition code. This is a way of telling the library: "give me a tensor that has these many columns and you compute the appropriate number of rows that is necessary to make this happen". ( Note that you can extend this to tensors with more dimensions. If there is any situation that you don't know how many rows you want but are sure of the number of columns, then you can specify this with a -1. Reshaping the tensor a to a 3 x 5 tensor would not be appropriate. Note that after the reshape the total number of elements need to remain the same. To reshape this tensor to make it a 4 x 4 tensor, use: a = a.view(4, 4) Given a tensor a with 16 elements: import torch These two lines were just to illustrate how we can still 'get back' to x from a slice of x, we can recover the apparent content by changing the tensor's metadata.View() reshapes the tensor without copying memory, similar to numpy's reshape(). Same with z: > torch.as_strided(z, size=x.shape, stride=x.stride())īoth will return a copy of x because torch.as_strided is allocating memory for the newly created tensor. In fact, you can even go from y to x's representation using torch.as_strided: > torch.as_strided(y, size=x.shape, stride=x.stride()) Here again, you notice that x.data_ptr is the same as z.data_ptr. > z.shape, z.stride(), x_ptr = z.data_ptr() Permuting the first two axes then slicing on the first one: > z = x.permute(1, 0, 2) > y.shape, y.stride(), x_ptr = y.data_ptr()Īs you can see, x and x shared the same storage. We have kept the data pointer for our 'base' tensor in x_ptr: Let us look at a simple example: > x = torch.ones(2,3,4).transpose(0,1) how you interact with that buffer (strides and shape) changes. Those two are essentially the same, the underlying data storage buffer is kept the same, only the metadata i.e. Since permute doesn't affect the underlying memory layout, both operations are essentially equivalent. but in fact share the identical storage buffer with the original tensor. TLDR Slices seemingly contain less information. (x.shape, 3, ADDITIONAL_DIM, x.shape) + x.shape In terms of memory locality it would make any difference to have a,b,c = torch.zeros( Will create three tensors with the same batch size x.shape. (3, x.shape, ADDITIONAL_DIM, x.shape) + x.shape When I create additional variables I am creating them with torch.zeros and then transposing, so that the largest stride goes to the axis 1, as well. Print(x.stride(), u.stride(), v.stride()) Results from arithmetic operations seems to keep the same memory layout x = torch.ones(2,3,4).transpose(0,1) My model calls a function that becomes simpler if I have another dimension in the first axis, so that I can use x instead of x. I know that usually the batch dimension is axis zero, and I imagine this has a reason: The underlying memory for each item in the batch is contiguous. ![]()
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |