Very cool. I would've used this a lot in a past life. It's interesting how many different ways there are to manage cron jobs, too. I've seen:
Place 1: Cron jobs edited on the box via vim, and every night they're scraped and off there are diffs, they're committed. Kind of like reverse management. It was actually a pretty elegant solution for the environment it was in, where devs just could not be convinced to not just edit the crontab directly.
Place 2: no cron jobs, it's all either airflow tasks or a bespoke managed clustering cron-like thing.
Place 3: all k8s scheduled job pods, conceptually the same but somehow less enjoyable.
Honestly I liked the airflow version the best, python is just better than bash, and often cron jobs end up developing dependencies anyway. Plus Python lets you wire in many niceties.
Very cool. I would've used this a lot in a past life. It's interesting how many different ways there are to manage cron jobs, too. I've seen:
Place 1: Cron jobs edited on the box via vim, and every night they're scraped and off there are diffs, they're committed. Kind of like reverse management. It was actually a pretty elegant solution for the environment it was in, where devs just could not be convinced to not just edit the crontab directly.
Place 2: no cron jobs, it's all either airflow tasks or a bespoke managed clustering cron-like thing.
Place 3: all k8s scheduled job pods, conceptually the same but somehow less enjoyable.
Honestly I liked the airflow version the best, python is just better than bash, and often cron jobs end up developing dependencies anyway. Plus Python lets you wire in many niceties.