Django + celery + beat 执行定时任务

django zhuoyuebiji -- 2019-12-17 0:26 其他

开发环境

python 3.6.8
django 1.11
celery 4.3.0
django-celery-results 1.1.2
django-celery-beat 1.5.0

安装 django + celery + results

安装 django-celery-beat

pip install django-celery-beat

配置 settings.py

INSTALLED_APPS = [
    # ...
    'django_celery_results',  # 查看 celery 执行结果
    'django_celery_beat',  # pip install django-celery-beat
]

生成 Django-celery-beat 关联表

python manage.py migrate

(joyoo) yinzhuoqundeMacBook-Pro:joyoo yinzhuoqun$ python manage.py migrate
raven.contrib.django.client.DjangoClient: 2019-12-16 23:01:02,792 /Users/yinzhuoqun/.pyenv/joyoo/lib/python3.6/site-packages/raven/base.py [line:213] INFO Raven is not configured (logging is disabled). Please see the documentation for more information.
Operations to perform:
  Apply all migrations: admin, auth, blog, captcha, contenttypes, django_celery_beat, django_celery_results, logger, photo, sessions, sites, user, users
Running migrations:
  Applying django_celery_beat.0001_initial... OK
  Applying django_celery_beat.0002_auto_20161118_0346... OK
  Applying django_celery_beat.0003_auto_20161209_0049... OK
  Applying django_celery_beat.0004_auto_20170221_0000... OK
  Applying django_celery_beat.0005_add_solarschedule_events_choices... OK
  Applying django_celery_beat.0006_auto_20180322_0932... OK
  Applying django_celery_beat.0007_auto_20180521_0826... OK
  Applying django_celery_beat.0008_auto_20180914_1922... OK
  Applying django_celery_beat.0006_auto_20180210_1226... OK
  Applying django_celery_beat.0006_periodictask_priority... OK
  Applying django_celery_beat.0009_periodictask_headers... OK
  Applying django_celery_beat.0010_auto_20190429_0326... OK
  Applying django_celery_beat.0011_auto_20190508_0153... OK

app 目录 tasks.py 添加 task

#!/usr/bin/env python3
# -*- coding: utf-8 -*-

"""
@author: yinzhuoqun
@site: http://zhuoqun.info/
@email: yin@zhuoqun.info
@time: 2019/12/15 12:34 AM
"""

import json
import requests
import datetime

from celery import task

from django.core.cache import cache
from django.core.mail import send_mail
from django.utils import timezone

from .models import TodayViewPage


# from django.core.mail import EmailMultiAlternatives


@task
def task_send_dd_text(url, msg, atMoblies, atAll="flase"):
    """
    发送钉钉提醒
    :param url:
    :param msg:
    :param atMoblies:
    :param atAll:
    :return:
    """
    body = {
        "msgtype": "text",
        "text": {
            "content": msg
        },
        "at": {
            "atMobiles": atMoblies,
            "isAtAll": atAll
        }
    }
    headers = {'content-type': 'application/json',
               'User-Agent': 'Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:22.0) Gecko/20100101 Firefox/22.0'}
    r = requests.post(url, headers=headers, data=json.dumps(body))
    # print(r.text)
    return r.text


@task
def task_send_mail(*args, **kwargs):
    """
    django 的 发送邮件，支持 html，html_message="html 内容"
    :param args:
    :param kwargs:
    :return:
    """
    return send_mail(*args, **kwargs)


@task
def task_save_view_page():
    """
    保存前一日访问量
    :return:
    """
    today = datetime.date.today()
    today_page_view = cache.get("today_page_view_%s" % (today.day - 1))
    if today_page_view:
        yesterday = datetime.date.today() + datetime.timedelta(days=-1)
        page_view_query = TodayViewPage.objects.filter(today=yesterday)
        if page_view_query.exists():
            page_view_obj = page_view_query.first()
            page_view_obj.view_page = today_page_view
            page_view_obj.save()
        else:
            page_view_obj = TodayViewPage(
                today=yesterday,
                view_page=today_page_view
            )
            page_view_obj.save()

        return "view page save success"

    return "view page save failed"

管理后台配置 Periodic tasks

启动 celery worker

# 项目根目录终端执行（joyoo 项目名称）

celery -A joyoo worker -l info

# 守护进程

/root/.virtualenvs/blog/bin/celery multi start w1 -A joyoo -l info --logfile=./celerylog.log

启动 celery beat

# 项目根目录终端执行（joyoo 项目名称）

celery -A joyoo beat -l info --scheduler django_celery_beat.schedulers:DatabaseScheduler

celery beat 日志

(joyoo) yinzhuoqundeMacBook-Pro:joyoo yinzhuoqun$ celery -A joyoo beat -l info --scheduler django_celery_beat.schedulers:DatabaseScheduler
celery beat v4.3.0 (rhubarb) is starting.
raven.contrib.django.client.DjangoClient: 2019-12-17 00:18:36,990 /Users/yinzhuoqun/.pyenv/joyoo/lib/python3.6/site-packages/raven/base.py [line:213] INFO Raven is not configured (logging is disabled). Please see the documentation for more information.
__    -    ... __   -        _
LocalTime -> 2019-12-17 00:18:38
Configuration ->
    . broker -> redis://127.0.0.1:6379/0
    . loader -> celery.loaders.app.AppLoader
    . scheduler -> django_celery_beat.schedulers.DatabaseScheduler

    . logfile -> [stderr]@%INFO
    . maxinterval -> 5.00 seconds (5s)
[2019-12-17 00:18:38,076: INFO/MainProcess] beat: Starting...
[2019-12-17 00:18:38,076: INFO/MainProcess] Writing entries...
[2019-12-17 00:18:43,105: INFO/MainProcess] Writing entries...
[2019-12-17 00:19:23,155: INFO/MainProcess] DatabaseScheduler: Schedule changed.
[2019-12-17 00:19:23,155: INFO/MainProcess] Writing entries...
[2019-12-17 00:20:00,008: INFO/MainProcess] Scheduler: Sending due task celery.save_view_page (blog.tasks.task_save_view_page)
[2019-12-17 00:21:45,165: INFO/MainProcess] Writing entries...
[2019-12-17 00:24:45,410: INFO/MainProcess] Writing entries...
[2019-12-17 00:27:45,649: INFO/MainProcess] Writing entries...
[2019-12-17 00:30:45,878: INFO/MainProcess] Writing entries...
[2019-12-17 00:32:05,981: INFO/MainProcess] DatabaseScheduler: Schedule changed.
[2019-12-17 00:32:05,981: INFO/MainProcess] Writing entries...
[2019-12-17 00:33:00,001: INFO/MainProcess] Scheduler: Sending due task celery.save_view_page (blog.tasks.task_save_view_page)
[2019-12-17 00:33:50,066: INFO/MainProcess] Writing entries...

celery worker 日志

(joyoo) yinzhuoqundeMacBook-Pro:joyoo yinzhuoqun$ celery -A joyoo worker -l info
raven.contrib.django.client.DjangoClient: 2019-12-17 00:18:48,673 /Users/yinzhuoqun/.pyenv/joyoo/lib/python3.6/site-packages/raven/base.py [line:213] INFO Raven is not configured (logging is disabled). Please see the documentation for more information.
 
 -------------- celery@yinzhuoqundeMacBook-Pro.local v4.3.0 (rhubarb)
---- **** ----- 
--- * ***  * -- Darwin-18.6.0-x86_64-i386-64bit 2019-12-17 00:18:49
-- * - **** --- 
- ** ---------- [config]
- ** ---------- .> app:         joyoo:0x10bb887f0
- ** ---------- .> transport:   redis://127.0.0.1:6379/0
- ** ---------- .> results:     
- *** --- * --- .> concurrency: 12 (prefork)
-- ******* ---- .> task events: OFF (enable -E to monitor tasks in this worker)
--- ***** ----- 
 -------------- [queues]
                .> celery           exchange=celery(direct) key=celery
                

[tasks]
  . blog.tasks.task_save_view_page
  . blog.tasks.task_send_dd_text
  . blog.tasks.task_send_mail

[2019-12-17 00:18:50,413: INFO/MainProcess] Connected to redis://127.0.0.1:6379/0
[2019-12-17 00:18:50,423: INFO/MainProcess] mingle: searching for neighbors
[2019-12-17 00:18:51,447: INFO/MainProcess] mingle: all alone
[2019-12-17 00:18:51,462: WARNING/MainProcess] /Users/yinzhuoqun/.pyenv/joyoo/lib/python3.6/site-packages/celery/fixups/django.py:202: UserWarning: Using settings.DEBUG leads to a memory leak, never use this setting in production environments!
  warnings.warn('Using settings.DEBUG leads to a memory leak, never '
[2019-12-17 00:18:51,463: INFO/MainProcess] celery@yinzhuoqundeMacBook-Pro.local ready.
[2019-12-17 00:20:00,030: INFO/MainProcess] Received task: blog.tasks.task_save_view_page[b375473b-5887-4cb4-8ddd-2d8cfdec79cc]  
[2019-12-17 00:20:00,070: INFO/ForkPoolWorker-8] Task blog.tasks.task_save_view_page[b375473b-5887-4cb4-8ddd-2d8cfdec79cc] succeeded in 0.037283750003552996s: '访问量保存成功'
[2019-12-17 00:33:00,003: INFO/MainProcess] Received task: blog.tasks.task_save_view_page[beb2a503-a116-4b8c-855a-0b645aa4079b]  
[2019-12-17 00:33:00,047: INFO/ForkPoolWorker-9] Task blog.tasks.task_save_view_page[beb2a503-a116-4b8c-855a-0b645aa4079b] succeeded in 0.04218383401166648s: '访问量保存成功'

文章部分资料可能来源于网络，如有侵权请告知删除。谢谢！

前一篇：学习过程的四个阶段
下一篇： django 组装表名查询数据（动态表名、组合表名）

阅读目录