Help Needed With TypeError: strptime() argument 1 must be str, not None

Hi all,

Firstly, I posted this yesterday - apologies for posting again today. I’m doing so because it seems this site is dead at the moment (my post received just 11 views and no replies). Although, perhaps I was unlucky or didn’t explain the problem concisely enough. Anything is possible, and I obviously mean no offence to those that give their time freely here. This problem has been driving me nuts, hence also why I’ve posted it again. Thanks.

The bug encountered is an unusual one, because this worked about a month - 6 weeks ago. According to the traceback, “stamp” - a uniquely named first argument - is None. The bot (the program) connects fine, but crashes once a message is sent as it tries to write to the database. I’ve tried a few different things, but cannot seem to find the appropriate fix to get this working, including deleting all records, renaming the database and attempting a few different “if”, “==” and “is not” statements.

The program is a chatbot for accessing IRC and allowing interaction with viewers through commands and messaging. That particular block of code is attempting to essentially update a SQL database setting the fields: username, userId, messagesSent, coinLock and coins. The parameters used are in text, text, integer, text (timestamp) and integer, respectively. The code for cursor fetchall etc (SQL stuff) is in a separate module.

I’d just like to add I’m not the most competent Python programmer - I only have perhaps 2 - 3 months experience and this is technically from an open-source tutorial, which was incidentally my first ever tutorial. A bit intense for a first tutorial, I know! Code and traceback below. Just mentioning, I’ve deleted the beginning of the filepaths containing personal files & username since this is a public post.

Thanks everyone, and I hope this is enough information!

def update_records(bot, user):
    db.execute("UPDATE users SET UserName = ?, MessagesSent = MessagesSent + 1 WHERE UserID = ?",
        user ["name"].lower(), user["id"])

    stamp = db.field("SELECT CoinLock FROM users WHERE UserID = ?", user["id"])

    if datetime.strptime(stamp,"%Y-%m-%d %H:%M:%S") < datetime.utcnow():
        coinlock = (datetime.utcnow()+timedelta(seconds=60)).strftime("%Y-%m-%d %H:%M:%S")
        
        db.execute("UPDATE users SET Coins = Coins + ?, CoinLock = ? WHERE UserID = ?",
            randint(1, 5), coinlock, user["id"])

Traceback (most recent call last):
line 76, in
bot.start()
Python\Python38-32\lib\site-packages\irc\bot.py", line 353, in start
super(SingleServerIRCBot, self).start()
Python\Python38-32\lib\site-packages\irc\client.py", line 1200, in start
self.reactor.process_forever()
Python\Python38-32\lib\site-packages\irc\client.py", line 844, in process_forever
consume(repeatfunc(one))
Python\Python38-32\lib\site-packages\more_itertools\recipes.py", line 137, in consume
deque(iterator, maxlen=0)
Python\Python38-32\lib\site-packages\irc\client.py", line 825, in process_once
self.process_data(in_)
Python\Python38-32\lib\site-packages\irc\client.py", line 790, in process_data
conn.process_data()
Python\Python38-32\lib\site-packages\irc\client.py", line 268, in process_data
self._process_line(line)
Python\Python38-32\lib\site-packages\irc\client.py", line 299, in _process_line
handler(arguments, command, source, tags)
Python\Python38-32\lib\site-packages\irc\client.py", line 343, in _handle_message
self._handle_event(event)
Python\Python38-32\lib\site-packages\irc\client.py", line 376, in _handle_event
self.reactor._handle_event(self, event)
Python\Python38-32\lib\site-packages\irc\client.py", line 920, in _handle_event
result = handler.callback(connection, event)
Python\Python38-32\lib\site-packages\irc\client.py", line 1157, in _dispatcher
method(connection, event)
line 9, in inner
func(*args, **kwargs)
line 65, in on_pubmsg
react.process(bot, user, message)
line 15, in process
update_records(bot, user)
line 49, in update_records
if datetime.strptime(stamp,"%Y-%m-%d %H:%M:%S") < datetime.utcnow():
TypeError: strptime() argument 1 must be str, not None
[Finished in 12.4s]

The only thing that’s obvious here is that the None must have been returned by the db.field() call. Saying why that might be would require knowledge of your DB structure and the library you’re using for access. My guess would be that the CoinLock field hasn’t been initialized correctly.

1 Like

Thanks for the reply, airtower luna! It’s really appreciated.

The libraries in the module with the error include the following:

from collections import defaultdict
from datetime import datetime, timedelta
from random import randint
from re import search
from time import time

Here’s the database structure:

from sqlite3 import connect

cxn = connect("./files/database.db", check_same_thread=False)
cur = cxn.cursor()


def with_commit(func):
    def inner(*args, **kwargs):
        func(*args, **kwargs)
        commit()
    return inner


@with_commit
def build():
    scriptexec("./files/script.sql")


def commit():
    cxn.commit()


def close():
    cxn.close()


def field(command, *values):
    cur.execute(command, tuple(values))
    if (fetch := cur.fetchone()) is not None:
        return fetch[0]


def record(command, *values):
    cur.execute(command, tuple(values))
    return cur.fetchone()


def records(command, *values):
    cur.execute(command, tuple(values))
    return cur.fetchall()


def column(command, *values):
    cur.execute(command, tuple(values))
    return [item[0] for item in cur.fetchall()]


def execute(command, *values):
    cur.execute(command, tuple(values))


def multiexec(command, valueset):
    cur.executemany(command, valueset)


def scriptexec(filename):
    with open(filename, "r") as script:
        cur.executescript(script.read())

SQL code (in case):


CREATE TABLE IF NOT EXISTS users (
    UserID text PRIMARY KEY,
    UserName text,
    MessagesSent integer DEFAULT 0,
    Coins integer DEFAULT 0,
    CoinLock text DEFAULT CURRENT_TIMESTAMP,
    Warnings integer DEFAULT 0
);

If that’s not enough information, please let me know! Thanks.

This mean that if cur.fetchone() returns None your function doesn’t have an explicit return statement, which implicitly means return None. So: Your query didn’t return anything. You’ll have to investigate why that may be.

Side note: Your db module seems to have the database objects at module level, which is generally considered bad style, because it’ll lead to problems if the module is used for more than one thing in your application. I’m not sure if you even need a separate module, many of the functions seem to be just very thin and almost identical wrappers around the sqlite3 methods. If you do, you should wrap the DB in a class.

1 Like

Thanks for the reply. As I mentioned above, this was actually from a tutorial, so I really only half understand what’s going on here. Ha.

Can you explain why having db objects at module level is bad and perhaps explain an alternative? I have heard there can be injection issues with SQL. Also, there are other modules which use the database, which are economy and games. So, I’m assuming that’s why the database was programmed this way initially and not placed in the same module as others.

What would wrapping the db in a class improve? I’m assuming less processing and memory?

Apologies for the questions - as mentioned above I’m quite new to Python. I’m actually from a visual scripting and game scripting background, so I do understand programming concepts, but I’m far from an expert.

Thanks again.

Module code is generally (with very rare exceptions) supposed to be reusable. That means if you import the same module in entirely different parts of an application (or maybe some library you use imports it, too) those separate uses of the same module must not disturb each other.

The db module you have opens the database file when it is imported, which breaks that rule: All code that uses the module will use the exact same database. That may work for a simple example, but it’s error prone and bad style in general, so I’d recommend doing it right from the beginning. And that’s where we get back to using a class.

A class would let you isolate different database instances. Before using the database you’d have to create an object of the class that represents that specific database (e.g. by passing the filename of the SQLite file). Then the code that uses that specific database can make accesses through that object, and code that uses another database would create another object representing that other database, and both wouldn’t get in the way of each other.

That’s also what the Python sqlite3 module does (also in your code): Calling sqlite3.connect() creates an object, and all accesses through that object will use the database it is connected to. At the same time you can create other, independent objects to access other databases.

So the simplest thing you could do would be to just use that sqlite3.Connection object instead of writing your own class. The main reason not to do that would be if you want to eventually support other database types, and you want to handle the differences between them in your own module, so the rest of your code doesn’t have to.

Yes, and if you’re going to use databases you definitely should read up on that. Highly simplified: If you use data from untrusted sources (e.g. stuff people type on IRC, including their names) in your database queries, someone could try to include SQL control sequences in that data. If you don’t escape the incoming data correctly those control sequences will likely be executed. Here’s a humorous example:

1 Like

Thanks for the explanations. It’s all interesting stuff and something I definitely need to look up. I’m sure there are some great tutorials somewhere for the SQL and database stuff you mentioned, so I’ll go check 'em out and hopefully implement the stuff you recommended, as well as that fix for the timestamp issue.

Thanks again.

1 Like

Hi again,

Thanks again for your assistance on the 23rd of September Airtower - you were great. Anyway, since last time, I’ve managed to wrap the database module in a class, using the super().init method (and I believe it works, since the program behaves the same). But, unfortunately, I’ve been unable to ascertain what’s happening with the timestamp error previously discussed.

Looking at the entire program, timestamp and coinlock are only used in a single module. I know you mentioned something about the fetchone part of the db not having an explicit return statement, which is fine. But if you were faced with this error, where do you believe the update records function can be adjusted and in what way? Since I’ve mostly developed in Unreal 3 and 4’s scripting language, I’m used to simply having very visual, more manageable outcomes than the standard IDE, and having stuff happen dynamically when testing. Ha.

I realise that’s a lot to ask, so thanks a lot if you attempt to answer. Also, I’m no expert at debugging - which I know is a serious shortfall - so any recommendations on best practice? I mostly use Liclipse with PyDev, if that helps.

Thank you! :smile:

The important part is that the fetchone function returns None if there is no result for your query. So the record you are asking for doesn’t exist. I don’t know why the user ID isn’t in the DB, but you need to handle that situation. Could be that you need to create the user with reasonable defaults, or that it’s the result of some other bug, depending on how your application is structured.

One thing I noticed is that there’s an update using the same user ID. If that’s what is supposed to create the entry if it doesn’t exist yet you might need to commit it before querying. Databases aren’t exactly my focus, though. :wink:

Not sure if I’d call it best practice, but my process is usually:

  • Add a lot of logging/debug output, anything that could possibly be helpful. Intermediate data is important because it helps you figure out where exactly the problem is. For example in your case: Is the entry really missing from the DB, or is something wrong with the query?
  • Write a test (or test script) for the problem. Again, testing small parts helps you figure out where a problem is.
  • Use whatever other tools can give me missing information. For my networking stuff that often capturing packets and using Wireshark for analysis, for you it might be opening the SQLite DB with another tool (e.g. DB Browser for SQLite) and checking what’s actually in there.
  • Stare at the code and gathered information until something clicks.

And then repeat until I have figured out what’s wrong.

1 Like

Hey, no worries. You really were helpful and seem more active on here than most, which is wonderful for those who need advice. You really seem to know your stuff, too. If Github isn’t already, they should be paying ya. Ha.

How would you recommend I would create the user with reasonable defaults? I had to ask, sorry!
Whereabouts in the code is the “update using the same user ID?” I’m assuming it’s one of the Setters.
Yeah, databases aren’t my thing either! The first one I ever made was using Access with Visual Basic during School, for printing letters or something with unique addresses. This is my second, and I didn’t even originally write it, so my bemusement probably makes sense.

  • Can you give me a rundown on intermediate data and possibly a rundown on logging? Are you referring specifically to print() functions with if, else and else if statements, or is there something else there? All I ever used in Unreal was print and ‘pray’. Haha.
  • I’ve never written a test script sadly, but have written psuedocode and test plans before. What’s the basic principle of those / structure?
  • I actually already use DB Browser - thanks for the link, though. Interestingly, there used to be records in there in all columns, then when the code started failing, they remained fine, but wouldn’t update. Since then, I’ve deleted the table and there have been none more written; it remains completely empty, which is odd.
  • Staring has gotta be a winner. Ha.

Thanks again!

The good news, is I’ve got the program running and updating the database after rolling back the code, sorta speak (I basically just copied and pasted an older version in to the module). I’ve been having problems with the bot sending tons of messages since the update, but I believe I’ve solved it by removing the “bot” parameter from the update records function (as it was basically saying “Hi Bot” and posting about 3 “hello user” messages when online and a command executed even though the IDE said the parameter was unused) and using db.execute after the if datetime.strptime statement with the necessary variables. That was a very long sentence, I know!

The parent and child class works great, too, so thanks for letting me know about the premise of wrapping the database module in a class!

It seems the fundamental problem was the lack of a proper update to the records, or there was something more subtle missing in the “react” module. If I find it in my side-by-side view, I’ll let ya know later! And if heaven for bid if something else goes wrong, I’ll be right back.

I’d appreciate some advice related to my latest questions regardless, though!

Thanks again!

Updated code:

def update_records(user):
    
    db.execute("UPDATE users SET UserName = ?, MessagesSent = MessagesSent + 1 WHERE UserID = ?",
        user ["name"].lower(), user["id"])
    
    stamp = db.field("SELECT CoinLock FROM users WHERE UserID = ?",
        user["id"])
    
    if datetime.strptime(stamp, "%Y-%m-%d %H:%M:%S") < datetime.utcnow():
        coinlock = (datetime.utcnow()+timedelta(seconds=60)).strftime("%Y-%m-%d %H:%M:%S")
        
        db.execute("UPDATE users SET UserName = ?, MessagesSent = MessagesSent + 1 WHERE UserID = ? SET Coins = Coins + ?, CoinLock = ? WHERE UserID = ?", 
        user ["name"].lower(), user["id"]) (randint(1, 5), coinlock, user["id"])

Update: The code above worked the first time, but has a syntax error after the first use, which is odd. I’ve rolled back again to this version, so I’m assuming there’s something subtle somewhere. Also, the bot parameter was actually needed for a process funtion at the top of the module, so that’s probably why it crashed! It works fine now. I’ll test it more, though.

Older version:

def update_records(bot, user):
    db.execute("UPDATE users SET UserName = ?, MessagesSent = MessagesSent + 1 WHERE UserID = ?",
        user ["name"].lower(), user["id"])

    stamp = db.field("SELECT CoinLock FROM users WHERE UserID = ?", user["id"])

    if datetime.strptime(stamp,"%Y-%m-%d %H:%M:%S") < datetime.utcnow():
        coinlock = (datetime.utcnow()+timedelta(seconds=60)).strftime("%Y-%m-%d %H:%M:%S")
        
        db.execute("UPDATE users SET Coins = Coins + ?, CoinLock = ? WHERE UserID = ?",
            randint(1, 5), coinlock, user["id"])
1 Like

That really requires more thought about your application design than I have. The key point is: If your application encounters a user for who has no DB entry make sure to create it before calling anything that assumes there is. Or make nothing rely on the user entry existing, and creates it as needed. :wink:

In your update_records function there’s an UPDATE just before the query that returns None.

print() works in many cases, no if/else needed. In a more complex application it might be better to use the logging module, and e.g. attach information about line and file or module. Using logging also has the advantage that you can keep the debug calls in production code, and just increase the log level if you encounter issues.

The important part is that you get the details you need about how the data moves and changes through your application. Having __str__() or __repr__() methods on your object that provide internal details makes this a lot simpler, as do custom formatting functions for objects from modules outside your control.

By test script I just mean a script that does things to try out the function(s) you want to test/debug. For example yesterday I wanted to know if an HTTPS server correctly reacts with an alert when sent plaintext, and the same when mixing plaintext into a TLS connection. So I wrote a script that does that and checks the results. A big advantage of that is that I can easily repeat and adjust, which takes much less work than manually trying things.

The boundary between such a script and more formalized tests isn’t clear cut, you could do automated tests by running a collection of scripts. I later took the script apart and turned it into a test inside my custom testing framework (here’s the result, if you’re interested).

Python includes a test framework in the unittest module, I highly recommend using it, and also running tests automatically, no matter which kind of tests you use. When you have something that doesn’t work a good idea is to write a test for the expected behavior, and then figure out why it doesn’t work. Sort of the debug version of test driven development. :grin:

Thanks! Maybe I should eventually figure out the tax implications of enabling that “sponsor” button… :thinking:

1 Like

Ah, I know the update records bit you were on about. I think last time I removed it, it crashed the bot. I’ll have another fiddle around, though.

That script is really interesting. If I’m honest, I only half know what’s going on (the socket, host and port stuff, mostly) but it looks cool. I might peruse it in more detail some other time. Thanks for sharing it.

I’ve heard about unittest, but keep forgetting about it!

Ah, I’m new to Github and didn’t know there was a sponsor button. You should enable it if you’re working hard to help so many people out, but I get what you’re saying about the taxman. Also, taxes bite. Heh.

Thanks again.

1 Like