[SOLVED] pytorch RuntimeError: Expected object of scalar type Double but got scalar type Float

Issue

I am trying to implement a custom dataset for my neural network. But got this error when running the forward function. The code is as follows.

import torch
import torch.nn as nn
import torch.nn.functional as F
from torch.utils.data import Dataset, DataLoader
import numpy as np

class ParamData(Dataset):
    def __init__(self,file_name):
        self.data = torch.Tensor(np.loadtxt(file_name,delimiter = ','))    #first place
    def __len__(self):
        return self.data.size()[0]
    def __getitem__(self,i):
        return self.data[i]

class Net(nn.Module):
    def __init__(self,in_size,out_size,layer_size=200):
        super(Net,self).__init__()
        self.layer = nn.Linear(in_size,layer_size)
        self.out_layer = nn.Linear(layer_size,out_size)

    def forward(self,x):
        x = F.relu(self.layer(x))
        x = self.out_layer(x)
        return x

datafile = 'data1.txt'

net = Net(100,1)
dataset = ParamData(datafile)
n_samples = len(dataset)

#dataset = torch.Tensor(dataset,dtype=torch.double)   #second place
#net.float()                                          #thrid place

net.forward(dataset[0])         #fourth place

In the file data1.txt is a csv formatted text file containing certain numbers, and each dataset[i] is a size 100 by 1 torch.Tensor object of dtype torch.float64. The error message is as follows:

Traceback (most recent call last):
  File "Z:\Wrong.py", line 33, in <module>
    net.forward(dataset[0])
  File "Z:\Wrong.py", line 23, in forward
    x = F.relu(self.layer(x))
  File "E:\Python38\lib\site-packages\torch\nn\modules\module.py", line 532, in __call__
    result = self.forward(*input, **kwargs)
  File "E:\Python38\lib\site-packages\torch\nn\modules\linear.py", line 87, in forward
    return F.linear(input, self.weight, self.bias)
  File "E:\Python38\lib\site-packages\torch\nn\functional.py", line 1372, in linear
    output = input.matmul(weight.t())
RuntimeError: Expected object of scalar type Double but got scalar type Float for argument #2 'mat2' in call to _th_mm

It seems that I should change the dtype of the numbers in dataset to torch.double. I tried things like

• changing the line at the first place to self.data = torch.tensor(np.loadtxt(file_name,delimiter = ','),dtype=torch.double)

• changing the line at the fourth place to net.forward(dataset[0].double())

• uncommenting one of the two lines at the second or the thrid place

I think these are the solutions I have seen from similar questions, but they either give new errors or don’t do anything. What should I do?

---

Update: So I got it working by changing the first place to

self.data = torch.from_numpy(np.loadtxt(file_name,delimiter = ',')).float()

which is weird because it is exactly the opposite of the error message. Is this a bug? I’d still like some explaining.

Solution

Now that I have more experience with pytorch, I think I can explain the error message. It seems that the line

RuntimeError: Expected object of scalar type Double but got scalar type Float for argument #2 'mat2' in call to _th_mm

is actually refering to the weights of the linear layer when the matrix multiplication is called. Since the input is double while the weights are float, it makes sense for the line

output = input.matmul(weight.t())

to expect the weights to be double.

Answered By – ihdv

Answer Checked By – Cary Denson (BugsFixing Admin)