Page 1 of 1

Sync of public repositories (like GitHub) and the self-made

Posted: Mon Oct 21, 2013 5:35 am
by Ivan Denisov
I have made some trial work for check community idea about Redmine-GitHub sync.

The results:
This is Redmine repository with ODC-to-UTF8 converter: http://redmine.molpit.com/projects/bbcb ... repository
every 5 minutes it synchronizing with GitHub organization type repository: https://github.com/bbfw-center/bbcb

That allows to see the diff in web-interface for BlackBox binaries odc format, and work with public GitHub repository at the same time.

Please, do not look at my actions as forcing, that work is a demo for you to see the ability and how it can be done.

The same can be done with Bitbucket and Gitorious, so to be honest this do not help to choose public repository.

Re: Sync of public repositories (like GitHub) and the self-m

Posted: Mon Oct 21, 2013 8:18 am
by Josef Templ
Ivan, could you give us a bit of background information how this has been set up.
I mean the dealing with binary text files together with git diff. This is a great feature but
how does it work? Does it only work in redmine or can it also be included directly in GitHub or
Gitorious or whatever git hosting platform. Probably not. But why is it possible in redmine.
Also, can it be done with resource files such as forms? For example, under my local svn
I managed to map BlackBox resource files to the 'Info->Control List' text and compare the two texts.
(Unfortunately, however, the 'Control List' text does not contain all the information of a form,
but this can be seen as a technical detail problem.)

- Josef

Re: Sync of public repositories (like GitHub) and the self-m

Posted: Mon Oct 21, 2013 9:41 am
by Ivan Denisov
Small review of question history. Boris Ilov has suggested to try Redmine for BlackBox development, the idea was supported by Ilya Ermakov and I have tried to install and play with Redmine. Boris Ilov has suggested to find ruby programmer (Redmine is made on Ruby) for writing the hook for solving the problem of binary preview. That gives me idea, that the problem has ready solution and after googling I have found the article about git converters. The main idea is that git before comparing or annotating can apply any converter to binary files. Then Roman Miro has mentioned that such converter "odcread" had been already developed.

To use "odcread" in local repository as binary to text converter:
1. Download odcread, compile and install in folder /usr/local/bin/
2. In local repository make file .git/info/attributes with content *.odc diff=cp
3. In file .git/config add next three strings:

Code: Select all

[diff "cp"]
binary = true
textconv = /usr/local/bin/odcread
So, it is impossible to use in GitHub, Bitbucket or Gitorious, because they did not allowing to execute any software on their (server) side.

I tried to use such converter in my server there Redmine is installed now. But solution was working very unstable, some documents were not showed. First I have thought that the problem is in charsets and make parsing scripts for cutting them out, but this did not give expected results. It had take a month to find that the problem in Apache2 Phusion Passenger module. Thanks to Roman for suggested alternative, after I have set up Unicoron server, the problem is over.

The technical summary of redmine.molpit.com is here.
Redmine now is version 2.3.2 stable, Ruby 2.0.0-p247 (2013-06-27).

Apache Phusion Passenger module replaced by Unicorn HTTP server for Rack applications (4.6.3) via Apache2 Proxy module for serving of Redmine. The reason is the bug in the Passenger Iconv usage, that crushes the view of many diffs in repository.

The git binary-to-text converter is based on a 'odcread' application processed by python script for replacing of tabs with spaces.

Grack application is used to have access to git repository via SmartHTTP. Grack is serving by Apache Phusion Passenger (4.0.14) compiled from ruby environment
Josef Templ wrote:Also, can it be done with resource files such as forms? For example, under my local svn
I managed to map BlackBox resource files to the 'Info->Control List' text and compare the two texts.
(Unfortunately, however, the 'Control List' text does not contain all the information of a form,
but this can be seen as a technical detail problem.)
Technically it is possible but you need to improve converter. Now "odcread" do not supporting forms 'odc'. Peter Kushnir have made some alternative converter, but it is not returning text in standard output. Instead of that converted info is stored inside odc document. I think, we can continue developing Peter converter and use it in future instead of "odcread". However the problem is that TextView have GUI dependence, so it is difficult to use in server (I have tried :) already).