View Issue Details

IDProjectCategoryView StatusLast Update
0000477filegeneralpublic2016-02-02 19:08
ReporterÁlvaro Justen 
Assigned ToChristos Zoulas 
PrioritynormalSeverityminorReproducibilityhave not tried
Status resolvedResolutionfixed 
Product Version5.24 
Target VersionFixed in Version5.26 
Summary0000477: Add magic-python to Python Package Index (PyPI)
DescriptionIf I install python-magic on Debian I'll end up with your implementation (thanks, btw!) but I can't install it directly from PyPI. :-(
There are lots of "magic" implementations available on PyPI but none seems to be stable enough. Most of the implementations aren't on Debian so it will create problems for all Python packages that depends on it and want to be added to Debian also (that's my case with my library rows[https://github.com/turicas/rows]).
It'd be nice if your implementation is added to PyPI. It can be easily done by simply modifying your setup.py to include a real package name (instead of a description) and running `python setup.py bdist_wheel sdist upload` (after registering into PyPI).

Thanks for the great work!
TagsNo tags attached.

Relationships

Activities

Christos Zoulas

Christos Zoulas

2015-09-09 18:26

manager   ~0001050

It seems that there are a bunch of broken packages, magic, pymagic, that have all the names that are reasonable and that the pypi collection is not being curated, so it is not very useful to upload stuff there.
Christos Zoulas

Christos Zoulas

2015-09-09 18:26

manager   ~0001051

how can you get rid of stale packages in pypi so we can re-use the names?
Álvaro Justen

Álvaro Justen

2015-09-09 19:16

reporter   ~0001052

Thanks for your response, Christos!

I don't know if there is a way to take some package name on PyPI (unless you convince the owner to transfer the ownership for you).

I think you can start creating a package called `file-magic` (does not exist yet), put the library there and we'll have a way to install it from PyPI.
Then, I can help you contacting everybody that owns *magic* names on PyPI and also other libraries which use these libraries so people will know the new, official one. :-)
Álvaro Justen

Álvaro Justen

2015-10-06 20:39

reporter   ~0001089

Hello, Christos. Do you have any news on this issue?
Thanks.
Christos Zoulas

Christos Zoulas

2015-10-07 12:00

manager   ~0001090

Sorry no progress; indecision inertia.
Álvaro Justen

Álvaro Justen

2015-10-07 15:06

reporter   ~0001091

May I help you in the decision? ;-)
The pragmatic (and fastest) approach is just to put it on PyPI (I suggested the name "file-magic", which is not taken). The rest of the work I'll do, contacting the maintainers of other *magic* packages on PyPI etc.
If do you want I can modify the `setup.py` on GitHub to accomplish the needed changes and create a pull request.
Christos Zoulas

Christos Zoulas

2015-10-07 15:36

manager   ~0001092

just attach a diff here...
Álvaro Justen

Álvaro Justen

2015-11-22 08:46

reporter   ~0001149

Christos, sorry for this long delay.

I did some work on the Python package:

- Modified the setup.py so we can upload the package to PyPI;
- Created some helper functions (with automated tests) so usage of `magic` module will be pretty easy;
- Created little (still need more work on it) documentation about usage (helper-functions only) on README.md;
- Registered the name on PyPI (running `python setup.py register`): https://pypi.python.org/pypi/file-magic (I can add you as an owner also, just give me your username there).

I've pushed all my changes to a git branch called `feature/python-package` on my GitHub account. You can see the changes here: https://github.com/file/file/compare/master...turicas:feature/python-package?diff=split&expand=1&name=feature%2Fpython-package

I'll upload the patches related to the 4 commits here also.

I can upload this code directly to PyPI (if you accept my changes) or (if you want to do this) you can run: `python setup.py sdist bdist_wheel upload`.
Álvaro Justen

Álvaro Justen

2015-11-22 08:46

reporter  

0001-Modify-setup.py-so-we-can-upload-to-PyPI.patch (1,395 bytes)
From 86f69fcb2b15c2f3a40a3cccf41af6531b3608de Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?=C3=81lvaro=20Justen=20=28=40turicas=29?=
 <alvarojusten@gmail.com>
Date: Sun, 22 Nov 2015 05:47:12 -0200
Subject: [PATCH 1/4] Modify setup.py so we can upload to PyPI

---
 python/setup.py | 27 ++++++++++++++++++---------
 1 file changed, 18 insertions(+), 9 deletions(-)

diff --git a/python/setup.py b/python/setup.py
index 2c3b527..b754bfb 100644
--- a/python/setup.py
+++ b/python/setup.py
@@ -1,10 +1,19 @@
-# Python distutils build script for magic extension
-from distutils.core import setup
+# coding: utf-8
 
-setup(name = 'Magic file extensions',
-    version = '0.2',
-    author = 'Reuben Thomas',
-    author_email = 'rrt@sc3d.org',
-    license = 'BSD',
-    description = 'libmagic Python bindings',
-    py_modules = ['magic'])
+from setuptools import setup
+
+
+setup(name='file-magic',
+      version='0.2.0',
+      author='Reuben Thomas',
+      author_email='rrt@sc3d.org',
+      url='https://github.com/file/file',
+      license='BSD',
+      description='(official) libmagic Python bindings',
+      py_modules=['magic'],
+      classifiers = [
+          'Intended Audience :: Developers',
+          'License :: OSI Approved :: BSD License',
+          'Natural Language :: English',
+          'Topic :: Software Development :: Libraries :: Python Modules',
+      ])
-- 
2.6.2

Álvaro Justen

Álvaro Justen

2015-11-22 08:46

reporter  

0002-Add-helper-functions-and-its-tests.patch (3,946 bytes)
From 81be2aafad2ac895d6bff7a9792f22891ec4fbbb Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?=C3=81lvaro=20Justen=20=28=40turicas=29?=
 <alvarojusten@gmail.com>
Date: Sun, 22 Nov 2015 06:31:12 -0200
Subject: [PATCH 2/4] Add helper functions and its tests

---
 python/magic.py | 54 ++++++++++++++++++++++++++++++++++++++++++++++++++++--
 python/tests.py | 32 ++++++++++++++++++++++++++++++++
 2 files changed, 84 insertions(+), 2 deletions(-)
 create mode 100644 python/tests.py

diff --git a/python/magic.py b/python/magic.py
index 2c1c012..8cff29a 100644
--- a/python/magic.py
+++ b/python/magic.py
@@ -1,10 +1,13 @@
-#!/usr/bin/env python
+# coding: utf-8
+
 '''
 Python bindings for libmagic
 '''
 
 import ctypes
 
+from collections import namedtuple
+
 from ctypes import *
 from ctypes.util import find_library
 
@@ -32,7 +35,7 @@ MAGIC_PRESERVE_ATIME = PRESERVE_ATIME = 128
 MAGIC_RAW = RAW = 256
 MAGIC_ERROR = ERROR = 512
 MAGIC_MIME_ENCODING = MIME_ENCODING = 1024
-MAGIC_MIME = MIME = 1040
+MAGIC_MIME = MIME = 1040  # MIME_TYPE + MIME_ENCODING
 MAGIC_APPLE = APPLE = 2048
 
 MAGIC_NO_CHECK_COMPRESS = NO_CHECK_COMPRESS = 4096
@@ -47,6 +50,8 @@ MAGIC_NO_CHECK_ENCODING = NO_CHECK_ENCODING = 2097152
 
 MAGIC_NO_CHECK_BUILTIN = NO_CHECK_BUILTIN = 4173824
 
+FileMagic = namedtuple('FileMagic', ('mime_type', 'encoding', 'name'))
+
 
 class magic_set(Structure):
     pass
@@ -222,3 +227,48 @@ def open(flags):
     Flags argument as for setflags.
     """
     return Magic(_open(flags))
+
+
+# Objects used by `detect_from_` functions
+mime_magic = Magic(_open(MAGIC_MIME))
+mime_magic.load()
+none_magic = Magic(_open(MAGIC_NONE))
+none_magic.load()
+
+
+def _create_filemagic(mime_detected, type_detected):
+    mime_type, mime_encoding = mime_detected.split('; ')
+
+    return FileMagic(name=type_detected, mime_type=mime_type,
+                     encoding=mime_encoding.replace('charset=', ''))
+
+
+def detect_from_filename(filename):
+    '''Detect mime type, encoding and file type from a filename
+
+    Returns a `FileMagic` namedtuple.
+    '''
+
+    return _create_filemagic(mime_magic.file(filename),
+                             none_magic.file(filename))
+
+
+def detect_from_fobj(fobj):
+    '''Detect mime type, encoding and file type from file-like object
+
+    Returns a `FileMagic` namedtuple.
+    '''
+
+    file_descriptor = fobj.fileno()
+    return _create_filemagic(mime_magic.descriptor(file_descriptor),
+                             none_magic.descriptor(file_descriptor))
+
+
+def detect_from_content(byte_content):
+    '''Detect mime type, encoding and file type from bytes
+
+    Returns a `FileMagic` namedtuple.
+    '''
+
+    return _create_filemagic(mime_magic.buffer(byte_content),
+                             none_magic.buffer(byte_content))
diff --git a/python/tests.py b/python/tests.py
new file mode 100644
index 0000000..197a8fc
--- /dev/null
+++ b/python/tests.py
@@ -0,0 +1,32 @@
+# coding: utf-8
+
+import unittest
+
+import magic
+
+
+class MagicTestCase(unittest.TestCase):
+
+    filename = 'magic.py'
+    expected_mime_type = 'text/x-python'
+    expected_encoding = 'us-ascii'
+    expected_name = 'Python script, ASCII text executable'
+
+    def assert_result(self, result):
+        self.assertEqual(result.mime_type, self.expected_mime_type)
+        self.assertEqual(result.encoding, self.expected_encoding)
+        self.assertEqual(result.name, self.expected_name)
+
+    def test_detect_from_filename(self):
+        result = magic.detect_from_filename(self.filename)
+        self.assert_result(result)
+
+    def test_detect_from_fobj(self):
+        with open(self.filename) as fobj:
+            result = magic.detect_from_fobj(fobj)
+        self.assert_result(result)
+
+    def test_detect_from_content(self):
+        with open(self.filename) as fobj:
+            result = magic.detect_from_content(fobj.read(4096))
+        self.assert_result(result)
-- 
2.6.2

Álvaro Justen

Álvaro Justen

2015-11-22 08:46

reporter  

0003-Move-README-to-markdown-and-update-it.patch (1,661 bytes)
From 23be890e3e3f157737515faabdaf52fba0d54635 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?=C3=81lvaro=20Justen=20=28=40turicas=29?=
 <alvarojusten@gmail.com>
Date: Sun, 22 Nov 2015 06:31:34 -0200
Subject: [PATCH 3/4] Move README to markdown and update it

---
 python/README    | 13 -------------
 python/README.md | 29 +++++++++++++++++++++++++++++
 2 files changed, 29 insertions(+), 13 deletions(-)
 delete mode 100644 python/README
 create mode 100644 python/README.md

diff --git a/python/README b/python/README
deleted file mode 100644
index 8b9a2a7..0000000
--- a/python/README
+++ /dev/null
@@ -1,13 +0,0 @@
-This directory contains Python bindings to allow you to access the
-libmagic api. At the moment their status is "experimental".
-
-You can install the modules either with:
-
-$ python setup.py build
-$ python setup.py install
-
-or, if you have easy_install:
-
-$ easy_install .
-
-magic-python should work now!
diff --git a/python/README.md b/python/README.md
new file mode 100644
index 0000000..68d3af2
--- /dev/null
+++ b/python/README.md
@@ -0,0 +1,29 @@
+# `file-magic`: Python Bindings
+
+This library is a Python ctypes interface to `libmagic`.
+
+
+## Installing
+
+You can install `file-magic` either with:
+
+    python setup.py install
+    # or
+    easy_install .
+
+
+## Using
+
+    import magic
+
+    detected = magic.detect_from_filename('magic.py')
+    print 'Detected MIME type: {}'.format(detected.mime_type)
+    print 'Detected encoding: {}'.format(detected.encoding)
+    print 'Detected file type name: {}'.format(detected.name)
+
+
+## Developing/Contributing
+
+To run the tests:
+
+    python setup.py test
-- 
2.6.2

Álvaro Justen

Álvaro Justen

2015-11-22 08:47

reporter  

0004-Update-setup.py-with-new-version.patch (1,128 bytes)
From 14bedd7d51c30764e4eacfc572869fd73b0ea655 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?=C3=81lvaro=20Justen=20=28=40turicas=29?=
 <alvarojusten@gmail.com>
Date: Sun, 22 Nov 2015 06:31:56 -0200
Subject: [PATCH 4/4] Update setup.py with new version

---
 python/setup.py | 9 ++++++---
 1 file changed, 6 insertions(+), 3 deletions(-)

diff --git a/python/setup.py b/python/setup.py
index b754bfb..24ae182 100644
--- a/python/setup.py
+++ b/python/setup.py
@@ -1,16 +1,19 @@
 # coding: utf-8
 
+from __future__ import unicode_literals
+
 from setuptools import setup
 
 
 setup(name='file-magic',
-      version='0.2.0',
-      author='Reuben Thomas',
-      author_email='rrt@sc3d.org',
+      version='0.3.0',
+      author='Reuben Thomas, Álvaro Justen',
+      author_email='rrt@sc3d.org, alvarojusten@gmail.com',
       url='https://github.com/file/file',
       license='BSD',
       description='(official) libmagic Python bindings',
       py_modules=['magic'],
+      test_suite='tests',
       classifiers = [
           'Intended Audience :: Developers',
           'License :: OSI Approved :: BSD License',
-- 
2.6.2

Álvaro Justen

Álvaro Justen

2015-11-22 08:50

reporter  

0005-Add-a-CHANGELOG-for-the-Python-binding.patch (741 bytes)
From 6d02f2c9752cc31651f22abcf69b4c6ff6503b6d Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?=C3=81lvaro=20Justen=20=28=40turicas=29?=
 <alvarojusten@gmail.com>
Date: Sun, 22 Nov 2015 06:49:28 -0200
Subject: [PATCH 5/5] Add a CHANGELOG for the Python binding

---
 python/CHANGELOG.md | 8 ++++++++
 1 file changed, 8 insertions(+)
 create mode 100644 python/CHANGELOG.md

diff --git a/python/CHANGELOG.md b/python/CHANGELOG.md
new file mode 100644
index 0000000..2ec1266
--- /dev/null
+++ b/python/CHANGELOG.md
@@ -0,0 +1,8 @@
+# Python `file-magic` Log of Changes
+
+## `0.3.0`
+
+- Fix `setup.py` so we can upload to PyPI
+- Add function `detect_from_filename`
+- Add function `detect_from_fobj`
+- Add function `detect_from_content`
-- 
2.6.2

CVS Commit

CVS Commit

2015-11-23 20:41

developer   ~0001150


Module Name: file
Committed By: christos
Date: Mon Nov 23 20:41:43 UTC 2015

Modified Files:
    file/python: README magic.py setup.py
Added Files:
    file/python: CHANGELOG.md README.md tests.py

Log Message:
PR/477: Make python code pypi friendly.


Christos Zoulas

Christos Zoulas

2015-11-23 20:43

manager   ~0001151

I've committed the changes, my pypi username is zoulasc.
Álvaro Justen

Álvaro Justen

2015-11-25 16:27

reporter   ~0001157

Thanks! :D
I've added you as owner of package file-magic on PyPI: https://pypi.python.org/pypi/file-magic
Christos Zoulas

Christos Zoulas

2015-11-29 22:16

manager   ~0001164

Thanks, we are all set!
Álvaro Justen

Álvaro Justen

2016-02-02 13:05

reporter   ~0001200

Christos,

I thought you'd upload it to PyPI and ending didn't uploading it, sorry.
I did it yesterday and it's available there.
Here is a tutorial on its usage:
http://blog.justen.eng.br/2016/02/detecting-file-type-and-encoding-in-python.html

Can you please change the README on "python" folder to instruct users to install it by executing "pip install file-magic"?
CVS Commit

CVS Commit

2016-02-02 19:07

developer   ~0001203


Module Name: file
Committed By: christos
Date: Tue Feb 2 19:07:29 UTC 2016

Modified Files:
    file/python: README.md

Log Message:
PR/477: add pip install.


Christos Zoulas

Christos Zoulas

2016-02-02 19:08

manager   ~0001204

Committed to HEAD.

Issue History

Date Modified Username Field Change
2015-09-02 14:36 Álvaro Justen New Issue
2015-09-09 18:26 Christos Zoulas Note Added: 0001050
2015-09-09 18:26 Christos Zoulas Assigned To => Christos Zoulas
2015-09-09 18:26 Christos Zoulas Status new => assigned
2015-09-09 18:26 Christos Zoulas Note Added: 0001051
2015-09-09 18:26 Christos Zoulas Status assigned => feedback
2015-09-09 19:16 Álvaro Justen Note Added: 0001052
2015-09-09 19:16 Álvaro Justen Status feedback => assigned
2015-10-06 20:39 Álvaro Justen Note Added: 0001089
2015-10-07 12:00 Christos Zoulas Note Added: 0001090
2015-10-07 15:06 Álvaro Justen Note Added: 0001091
2015-10-07 15:36 Christos Zoulas Note Added: 0001092
2015-11-22 08:46 Álvaro Justen Note Added: 0001149
2015-11-22 08:46 Álvaro Justen File Added: 0001-Modify-setup.py-so-we-can-upload-to-PyPI.patch
2015-11-22 08:46 Álvaro Justen File Added: 0002-Add-helper-functions-and-its-tests.patch
2015-11-22 08:46 Álvaro Justen File Added: 0003-Move-README-to-markdown-and-update-it.patch
2015-11-22 08:47 Álvaro Justen File Added: 0004-Update-setup.py-with-new-version.patch
2015-11-22 08:50 Álvaro Justen File Added: 0005-Add-a-CHANGELOG-for-the-Python-binding.patch
2015-11-23 20:41 CVS Commit
2015-11-23 20:41 CVS Commit Note Added: 0001150
2015-11-23 20:41 CVS Commit Status assigned => confirmed
2015-11-23 20:41 CVS Commit Resolution open => fixed
2015-11-23 20:43 Christos Zoulas Note Added: 0001151
2015-11-23 20:43 Christos Zoulas Status confirmed => feedback
2015-11-25 16:27 Álvaro Justen Note Added: 0001157
2015-11-25 16:27 Álvaro Justen Status feedback => assigned
2015-11-29 22:16 Christos Zoulas Note Added: 0001164
2015-11-29 22:16 Christos Zoulas Status assigned => resolved
2015-11-29 22:16 Christos Zoulas Fixed in Version => 5.26
2016-02-02 13:05 Álvaro Justen Note Added: 0001200
2016-02-02 13:05 Álvaro Justen Status resolved => feedback
2016-02-02 13:05 Álvaro Justen Resolution fixed => reopened
2016-02-02 19:07 CVS Commit
2016-02-02 19:07 CVS Commit Note Added: 0001203
2016-02-02 19:07 CVS Commit Status feedback => confirmed
2016-02-02 19:07 CVS Commit Resolution reopened => fixed
2016-02-02 19:08 Christos Zoulas Note Added: 0001204
2016-02-02 19:08 Christos Zoulas Status confirmed => resolved