Here is model column:
title = Column(Unicode(100))
When I try to add a record to db with cyrillic letters in title I get an error:
UnicodeEncodeError:
latin-1′ codec can’t encode characters on position …
If i encode it with utf-8
title = request.POST['title'].encode('utf-8')
I get next error:
ProgrammingError: You must not use 8-bit bytestrings unless you use a text_factory that can interpret 8-bit btystrings (like text_factory = str). It is highly recommended that you instead just switch your application to Unicode strings.
When I decode utf-8
title = request.POST['title'].decode('utf-8')
There is new error
UnicodeDecodeError: ‘ascii’ codec can’t encode characters…
What to do?
EDIT:
I’m using Sqlite3, I thought it may be important.
EDIT 2: (code and traceback)
My code:
title = request.POST['title']
new_model = Model(
...
title = title,
...
)
DBSession.add(new_model)
DBSession.flush()
And Traceback (most recent call last):
File “/home/p/work/SUN/lib/python2.6/site-packages/pyramid_debugtoolbar-0.9.7-py2.6.egg/pyramid_debugtoolbar/panels/performance.py”, line 55, in resource_timer_handler
result = handler(request)
File “/home/p/work/SUN/lib/python2.6/site-packages/pyramid-1.3a3-py2.6.egg/pyramid/tweens.py”, line 20, in excview_tween
response = handler(request)
File “/home/p/work/SUN/lib/python2.6/site-packages/pyramid_tm-0.3-py2.6.egg/pyramid_tm/init.py”, line 61, in tm_tween
response = handler(request)
File “/home/p/work/SUN/lib/python2.6/site-packages/pyramid-1.3a3-py2.6.egg/pyramid/router.py”, line 164, in handle_request
response = view_callable(context, request)
File “/home/p/work/SUN/lib/python2.6/site-packages/pyramid-1.3a3-py2.6.egg/pyramid/config/views.py”, line 316, in rendered_view
result = view(context, request)
File “/home/p/work/SUN/lib/python2.6/site-packages/pyramid-1.3a3-py2.6.egg/pyramid/config/views.py”, line 426, in _requestonly_view
response = view(request)
File “/home/p/work/SUN/sunviver/sunviver/views/advert.py”, line 187, in create
return HTTPFound(location=new_advert.url())
File “/home/p/work/SUN/lib/python2.6/site-packages/pyramid-1.3a3-py2.6.egg/pyramid/httpexceptions.py”, line 444, in init
body_template=body_template, location=location, **kw)
File “/home/p/work/SUN/lib/python2.6/site-packages/pyramid-1.3a3-py2.6.egg/pyramid/httpexceptions.py”, line 213, in init
Response.init(self, status=status, **kw)
File “build/bdist.linux-i686/egg/webob/response.py”, line 141, in init
setattr(self, name, value)
File “build/bdist.linux-i686/egg/webob/descriptors.py”, line 112, in fset
value = value.encode(‘latin-1’)
And new_advert.url():
def url(self):
return '/%s/%s' % (self.id, self.title.replace(' ', '_'))
@J.F. Sebastian
Traceback (most recent call last):
File “/usr/lib/python2.6/wsgiref/handlers.py”, line 93, in run
self.result = application(self.environ, self.start_response)
File “/home/ponomar/v/ENV/lib/python2.6/site-packages/pyramid-1.3a3-py2.6.egg/pyramid/router.py”, line 187, in call
response = self.handle_request(request)
File “/home/ponomar/v/ENV/lib/python2.6/site-packages/pyramid_debugtoolbar-0.9.7-py2.6.egg/pyramid_debugtoolbar/toolbar.py”, line 157, in toolbar_tween
toolbar.process_response(response)
File “/home/ponomar/v/ENV/lib/python2.6/site-packages/pyramid_debugtoolbar-0.9.7-py2.6.egg/pyramid_debugtoolbar/toolbar.py”, line 55, in process_response
vars, request=request)
File “/home/ponomar/v/ENV/lib/python2.6/site-packages/pyramid-1.3a3-py2.6.egg/pyramid/renderers.py”, line 81, in render
return helper.render(value, None, request=request)
File “/home/ponomar/v/ENV/lib/python2.6/site-packages/pyramid-1.3a3-py2.6.egg/pyramid/renderers.py”, line 420, in render
result = renderer(value, system_values)
File “/home/ponomar/v/ENV/lib/python2.6/site-packages/pyramid-1.3a3-py2.6.egg/pyramid/mako_templating.py”, line 162, in call
reraise(MakoRenderingException(errtext), None, exc_info[2])
File “/home/ponomar/v/ENV/lib/python2.6/site-packages/pyramid-1.3a3-py2.6.egg/pyramid/mako_templating.py”, line 154, in call
result = template.render_unicode(**system)
File “/home/ponomar/v/ENV/lib/python2.6/site-packages/Mako-0.5.0-py2.6.egg/mako/template.py”, line 311, in render_unicode
as_unicode=True)
File “/home/ponomar/v/ENV/lib/python2.6/site-packages/Mako-0.5.0-py2.6.egg/mako/runtime.py”, line 660, in _render
**_kwargs_for_callable(callable_, data))
File “/home/ponomar/v/ENV/lib/python2.6/site-packages/Mako-0.5.0-py2.6.egg/mako/runtime.py”, line 692, in _render_context
_exec_template(inherit, lclcontext, args=args, kwargs=kwargs)
File “/home/ponomar/v/ENV/lib/python2.6/site-packages/Mako-0.5.0-py2.6.egg/mako/runtime.py”, line 718, in _exec_template
callable_(context, *args, **kwargs)
File “pyramid_debugtoolbar_templates_toolbar_mako”, line 117, in render_body
File “/home/ponomar/v/ENV/lib/python2.6/site-packages/pyramid_debugtoolbar-0.9.7-py2.6.egg/pyramid_debugtoolbar/panels/headers.py”, line 54, in content
vars, self.request)
File “/home/ponomar/v/ENV/lib/python2.6/site-packages/pyramid_debugtoolbar-0.9.7-py2.6.egg/pyramid_debugtoolbar/panels/init.py”, line 24, in render
return render(template_name, vars, request=request)
File “/home/ponomar/v/ENV/lib/python2.6/site-packages/pyramid-1.3a3-py2.6.egg/pyramid/renderers.py”, line 81, in render
return helper.render(value, None, request=request)
File “/home/ponomar/v/ENV/lib/python2.6/site-packages/pyramid-1.3a3-py2.6.egg/pyramid/renderers.py”, line 420, in render
result = renderer(value, system_values)
File “/home/ponomar/v/ENV/lib/python2.6/site-packages/pyramid-1.3a3-py2.6.egg/pyramid/mako_templating.py”, line 162, in call
reraise(MakoRenderingException(errtext), None, exc_info[2])
File “/home/ponomar/v/ENV/lib/python2.6/site-packages/pyramid-1.3a3-py2.6.egg/pyramid/mako_templating.py”, line 154, in call
result = template.render_unicode(**system)
File “/home/ponomar/v/ENV/lib/python2.6/site-packages/Mako-0.5.0-py2.6.egg/mako/template.py”, line 311, in render_unicode
as_unicode=True)
File “/home/ponomar/v/ENV/lib/python2.6/site-packages/Mako-0.5.0-py2.6.egg/mako/runtime.py”, line 660, in _render
**_kwargs_for_callable(callable_, data))
File “/home/ponomar/v/ENV/lib/python2.6/site-packages/Mako-0.5.0-py2.6.egg/mako/runtime.py”, line 692, in _render_context
_exec_template(inherit, lclcontext, args=args, kwargs=kwargs)
File “/home/ponomar/v/ENV/lib/python2.6/site-packages/Mako-0.5.0-py2.6.egg/mako/runtime.py”, line 718, in _exec_template
callable_(context, *args, **kwargs)
File “pyramid_debugtoolbar_panels_templates_headers_mako”, line 34, in render_body
File “/home/ponomar/v/ENV/lib/python2.6/site-packages/MarkupSafe-0.15-py2.6.egg/markupsafe/_native.py”, line 21, in escape
return Markup(unicode(s)
MakoRenderingException:
Traceback (most recent call last):
File “/home/ponomar/v/ENV/lib/python2.6/site-packages/pyramid-1.3a3-py2.6.egg/pyramid/mako_templating.py”, line 154, in call
result = template.render_unicode(**system)
File “/home/ponomar/v/ENV/lib/python2.6/site-packages/Mako-0.5.0-py2.6.egg/mako/template.py”, line 311, in render_unicode
as_unicode=True)
File “/home/ponomar/v/ENV/lib/python2.6/site-packages/Mako-0.5.0-py2.6.egg/mako/runtime.py”, line 660, in _render
**_kwargs_for_callable(callable_, data))
File “/home/ponomar/v/ENV/lib/python2.6/site-packages/Mako-0.5.0-py2.6.egg/mako/runtime.py”, line 692, in _render_context
_exec_template(inherit, lclcontext, args=args, kwargs=kwargs)
File “/home/ponomar/v/ENV/lib/python2.6/site-packages/Mako-0.5.0-py2.6.egg/mako/runtime.py”, line 718, in _exec_template
callable_(context, *args, **kwargs)
File “/home/ponomar/v/ENV/lib/python2.6/site-packages/pyramid_debugtoolbar-0.9.7-py2.6.egg/pyramid_debugtoolbar/templates/toolbar.mako”, line 61, in render_body
${panel.content()|n}
File “/home/ponomar/v/ENV/lib/python2.6/site-packages/pyramid_debugtoolbar-0.9.7-py2.6.egg/pyramid_debugtoolbar/panels/headers.py”, line 54, in content
vars, self.request)
File “/home/ponomar/v/ENV/lib/python2.6/site-packages/pyramid_debugtoolbar-0.9.7-py2.6.egg/pyramid_debugtoolbar/panels/init.py”, line 24, in render
return render(template_name, vars, request=request)
File “/home/ponomar/v/ENV/lib/python2.6/site-packages/pyramid-1.3a3-py2.6.egg/pyramid/renderers.py”, line 81, in render
return helper.render(value, None, request=request)
File “/home/ponomar/v/ENV/lib/python2.6/site-packages/pyramid-1.3a3-py2.6.egg/pyramid/renderers.py”, line 420, in render
result = renderer(value, system_values)
File “/home/ponomar/v/ENV/lib/python2.6/site-packages/pyramid-1.3a3-py2.6.egg/pyramid/mako_templating.py”, line 162, in call
reraise(MakoRenderingException(errtext), None, exc_info[2])
File “/home/ponomar/v/ENV/lib/python2.6/site-packages/pyramid-1.3a3-py2.6.egg/pyramid/mako_templating.py”, line 154, in call
result = template.render_unicode(**system)
File “/home/ponomar/v/ENV/lib/python2.6/site-packages/Mako-0.5.0-py2.6.egg/mako/template.py”, line 311, in render_unicode
as_unicode=True)
File “/home/ponomar/v/ENV/lib/python2.6/site-packages/Mako-0.5.0-py2.6.egg/mako/runtime.py”, line 660, in _render
**_kwargs_for_callable(callable_, data))
File “/home/ponomar/v/ENV/lib/python2.6/site-packages/Mako-0.5.0-py2.6.egg/mako/runtime.py”, line 692, in _render_context
_exec_template(inherit, lclcontext, args=args, kwargs=kwargs)
File “/home/ponomar/v/ENV/lib/python2.6/site-packages/Mako-0.5.0-py2.6.egg/mako/runtime.py”, line 718, in _exec_template
callable_(context, *args, **kwargs)
File “/home/ponomar/v/ENV/lib/python2.6/site-packages/pyramid_debugtoolbar-0.9.7-py2.6.egg/pyramid_debugtoolbar/panels/templates/headers.mako”, line 13, in render_body
${value|h}
File “/home/ponomar/v/ENV/lib/python2.6/site-packages/MarkupSafe-0.15-py2.6.egg/markupsafe/_native.py”, line 21, in escape
return Markup(unicode(s)
MakoRenderingException:
Traceback (most recent call last):
File “/home/ponomar/v/ENV/lib/python2.6/site-packages/pyramid-1.3a3-py2.6.egg/pyramid/mako_templating.py”, line 154, in call
result = template.render_unicode(**system)
File “/home/ponomar/v/ENV/lib/python2.6/site-packages/Mako-0.5.0-py2.6.egg/mako/template.py”, line 311, in render_unicode
as_unicode=True)
File “/home/ponomar/v/ENV/lib/python2.6/site-packages/Mako-0.5.0-py2.6.egg/mako/runtime.py”, line 660, in _render
**_kwargs_for_callable(callable_, data))
File “/home/ponomar/v/ENV/lib/python2.6/site-packages/Mako-0.5.0-py2.6.egg/mako/runtime.py”, line 692, in _render_context
_exec_template(inherit, lclcontext, args=args, kwargs=kwargs)
File “/home/ponomar/v/ENV/lib/python2.6/site-packages/Mako-0.5.0-py2.6.egg/mako/runtime.py”, line 718, in _exec_template
callable_(context, *args, **kwargs)
File “/home/ponomar/v/ENV/lib/python2.6/site-packages/pyramid_debugtoolbar-0.9.7-py2.6.egg/pyramid_debugtoolbar/panels/templates/headers.mako”, line 13, in render_body
${value|h}
File “/home/ponomar/v/ENV/lib/python2.6/site-packages/MarkupSafe-0.15-py2.6.egg/markupsafe/_native.py”, line 21, in escape
return Markup(unicode(s)
UnicodeDecodeError: ‘ascii’ codec can’t decode byte 0xd0 in position 7: ordinal not in range(128)
Encoding converts Unicode strings to byte strings. You need to go in the other direction,
decoding:
request.POST['title'].decode('utf-8')(assumingrequest.POST['title']is a UTF8-encodedstr).